-- Logs begin at Mon 2022-08-01 13:15:43 UTC, end at Mon 2022-08-01 14:27:43 UTC. -- Aug 01 13:51:58.999040 dv-cinder-845178-10-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 01 13:51:59.009729 dv-cinder-845178-10-iscsi devstack@keystone.service[80300]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 01 13:51:59.012472 dv-cinder-845178-10-iscsi devstack@keystone.service[80300]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 1 13:51:59 2022] *** Aug 01 13:51:59.012617 dv-cinder-845178-10-iscsi devstack@keystone.service[80300]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 01 13:51:59.012723 dv-cinder-845178-10-iscsi devstack@keystone.service[80300]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 01 13:51:59.012845 dv-cinder-845178-10-iscsi devstack@keystone.service[80300]: nodename: dv-cinder-845178-10-iscsi Aug 01 13:51:59.012942 dv-cinder-845178-10-iscsi devstack@keystone.service[80300]: machine: x86_64 Aug 01 13:51:59.013044 dv-cinder-845178-10-iscsi devstack@keystone.service[80300]: clock source: unix Aug 01 13:51:59.013145 dv-cinder-845178-10-iscsi devstack@keystone.service[80300]: pcre jit disabled Aug 01 13:51:59.013246 dv-cinder-845178-10-iscsi devstack@keystone.service[80300]: detected number of CPU cores: 4 Aug 01 13:51:59.013347 dv-cinder-845178-10-iscsi devstack@keystone.service[80300]: current working directory: / Aug 01 13:51:59.013448 dv-cinder-845178-10-iscsi devstack@keystone.service[80300]: detected binary path: /usr/bin/uwsgi-core Aug 01 13:51:59.013563 dv-cinder-845178-10-iscsi devstack@keystone.service[80300]: your processes number limit is 47680 Aug 01 13:51:59.013667 dv-cinder-845178-10-iscsi devstack@keystone.service[80300]: your memory page size is 4096 bytes Aug 01 13:51:59.013787 dv-cinder-845178-10-iscsi devstack@keystone.service[80300]: detected max file descriptor number: 2048 Aug 01 13:51:59.013787 dv-cinder-845178-10-iscsi devstack@keystone.service[80300]: lock engine: pthread robust mutexes Aug 01 13:51:59.014057 dv-cinder-845178-10-iscsi devstack@keystone.service[80300]: thunder lock: enabled Aug 01 13:51:59.014206 dv-cinder-845178-10-iscsi devstack@keystone.service[80300]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 01 13:51:59.014206 dv-cinder-845178-10-iscsi devstack@keystone.service[80300]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 01 13:51:59.054348 dv-cinder-845178-10-iscsi devstack@keystone.service[80300]: Python main interpreter initialized at 0x5560fe4e79c0 Aug 01 13:51:59.054348 dv-cinder-845178-10-iscsi devstack@keystone.service[80300]: python threads support enabled Aug 01 13:51:59.054348 dv-cinder-845178-10-iscsi devstack@keystone.service[80300]: your server socket listen backlog is limited to 100 connections Aug 01 13:51:59.054348 dv-cinder-845178-10-iscsi devstack@keystone.service[80300]: your mercy for graceful operations on workers is 90 seconds Aug 01 13:51:59.054770 dv-cinder-845178-10-iscsi devstack@keystone.service[80300]: mapped 403077 bytes (393 KB) for 2 cores Aug 01 13:51:59.054770 dv-cinder-845178-10-iscsi devstack@keystone.service[80300]: *** Operational MODE: preforking *** Aug 01 13:51:59.055089 dv-cinder-845178-10-iscsi devstack@keystone.service[80300]: *** uWSGI is running in multiple interpreter mode *** Aug 01 13:51:59.055089 dv-cinder-845178-10-iscsi devstack@keystone.service[80300]: spawned uWSGI master process (pid: 80300) Aug 01 13:51:59.055331 dv-cinder-845178-10-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 01 13:51:59.058403 dv-cinder-845178-10-iscsi devstack@keystone.service[80300]: spawned uWSGI worker 1 (pid: 80310, cores: 1) Aug 01 13:51:59.058403 dv-cinder-845178-10-iscsi devstack@keystone.service[80300]: spawned uWSGI worker 2 (pid: 80311, cores: 1) Aug 01 13:51:59.058403 dv-cinder-845178-10-iscsi devstack@keystone.service[80300]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 01 13:52:06.026956 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 13:52:06.026956 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: from cryptography.utils import int_from_bytes Aug 01 13:52:06.027954 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 13:52:06.027954 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: from cryptography.utils import int_from_bytes Aug 01 13:52:06.118705 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 01 13:52:06.118969 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 01 13:52:06.119273 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] command line args: [] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 01 13:52:06.119715 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 01 13:52:06.120022 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ================================================================================ {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 01 13:52:06.120427 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] admin_token = **** {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.120827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] config_dir = [] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.121119 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.121409 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] config_source = [] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.121694 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] debug = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.122011 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.123783 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.123783 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.123783 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.123783 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.124404 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] list_limit = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.124710 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] log_config_append = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.124989 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.125282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] log_dir = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.125565 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] log_file = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.125995 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.126400 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.126804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.127165 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.127447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.127723 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.128118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.128660 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.129038 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.129454 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.129768 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.130144 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.130845 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.130991 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.131361 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.132122 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.45/identity {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.132439 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] publish_errors = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.133098 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.133250 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.133434 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.133830 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.134198 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.134584 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] standard_threads = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.134942 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.135321 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.135604 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.135864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] use_journal = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.136112 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] use_json = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.136393 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] use_stderr = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.136670 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] use_syslog = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.137051 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.137577 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.137851 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.138325 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.138704 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.139095 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.139476 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.139870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.140244 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.140666 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.141084 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.141528 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.141898 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.142320 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.142995 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.143393 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.143774 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.144148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.144574 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.144982 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.145370 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.145750 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.146157 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.146567 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.146968 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.147365 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.147735 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.148087 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.148476 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.148834 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.149202 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.149568 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.149902 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.150248 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.150605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.150991 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.151363 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.151784 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.152110 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.152514 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.152876 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.153233 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.153583 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.153935 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.154285 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.154661 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.155055 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.155461 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.155810 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.156163 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.156568 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.157030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.157379 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.157738 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.158113 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.158489 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.158865 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.159245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.159644 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.160148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.160574 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.160935 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.161293 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.161641 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.162035 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.162391 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.162739 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.163101 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.163455 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.164029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.164418 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.164802 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.165211 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.165596 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.166004 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.166433 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.166867 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.167248 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.167622 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.168042 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.168492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.168880 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.169111 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 13:52:06.169111 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: from cryptography.utils import int_from_bytes Aug 01 13:52:06.169854 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.169948 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 13:52:06.169948 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: from cryptography.utils import int_from_bytes Aug 01 13:52:06.170190 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.170190 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.170452 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.170742 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.171125 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.171527 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.171914 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.172352 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.172892 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.173304 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.173746 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.174186 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.174665 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.175123 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.175563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.176023 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.176509 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.176998 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.177455 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.178019 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.178436 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.178893 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.179402 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] auth.external = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.179781 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.180386 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.180835 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.181288 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] auth.password = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.181677 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] auth.token = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.182022 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.182346 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.182687 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.183127 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.183606 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.184079 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.184544 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.184999 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] credential.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.185474 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.185943 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.186414 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.186873 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.187352 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.187796 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.188363 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.188849 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.189306 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.189788 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.190330 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.191177 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.191667 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.192129 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.192675 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.193126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] federation.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.193583 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.194069 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.194538 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.194981 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.195453 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.195914 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.196434 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.196920 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.197406 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.197864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.198352 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.198691 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.199118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.199604 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.200049 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.200550 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.201012 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.201444 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.201929 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.202376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.202849 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.203286 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.203742 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.204225 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.204742 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.205207 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.205669 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.206150 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.206621 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.207123 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.207579 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.208030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.208492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.208947 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.209383 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.209831 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.210291 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.210744 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.211241 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.211656 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.212118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.212616 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.213063 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.213693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.213989 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.214468 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.214881 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.215316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.215762 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.216212 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.216710 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.217161 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.217613 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.218086 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.218556 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.222458 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.222458 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.222458 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.222458 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.222458 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.222458 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.222458 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.222458 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.223500 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.223500 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.223500 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.224943 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.224943 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.224943 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.225320 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.225639 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.226086 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.226539 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.226989 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.227424 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.227887 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.228422 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.228852 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.229563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.230034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.230471 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.230938 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.231386 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.231836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.232327 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.232763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.233232 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.233703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.234168 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.234642 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.235142 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.235601 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.236071 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.236553 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.237005 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.237475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.237927 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.238427 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.238883 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.239340 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] resource.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.239811 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.240297 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.240773 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.241241 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.241718 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.242165 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.242625 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.243072 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.243524 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.243970 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] role.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.244726 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] role.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.244903 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.245544 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.246133 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.246359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.246738 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.247220 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.247753 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.248249 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.248751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.249194 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.249660 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.250134 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.250601 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.251102 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.251900 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.252446 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.252829 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.253298 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.253765 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.254239 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.254687 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.255169 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.255648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.255740 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 01 13:52:06.256004 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 01 13:52:06.256099 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.256386 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] command line args: [] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 01 13:52:06.256584 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.256875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 01 13:52:06.257027 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.257189 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ================================================================================ {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 01 13:52:06.257454 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.257628 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] admin_token = **** {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.257929 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.258073 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] config_dir = [] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.258384 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.258465 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.258703 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] config_source = [] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.258853 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.259039 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] debug = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.259305 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.259415 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.259804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.259961 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.260162 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.260328 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.260687 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.260807 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] token.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.261140 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.261257 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.261567 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] list_limit = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.261685 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.261886 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] log_config_append = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.262030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.262211 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.262507 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.262633 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] log_dir = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.262843 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] log_file = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.262961 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.263288 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.263408 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.263961 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.264607 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.264745 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.264865 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.265083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.265282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.266274 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.266513 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.266675 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.266851 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.267031 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.267155 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.267270 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.267705 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.267829 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.267943 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.268056 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.268168 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.268367 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.268367 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.268912 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.269115 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.269313 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.269506 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.269697 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.connection = **** {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.269970 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.270312 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.270312 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.270702 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.45/identity {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.270893 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.271093 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] publish_errors = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.271279 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.271463 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.271637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.271803 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.271967 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.272146 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.272385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.272553 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.272790 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.272964 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.273252 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.273447 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] standard_threads = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.273687 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.273883 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.274142 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.274322 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.274502 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.274678 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.274846 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] use_journal = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.275010 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.275181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] use_json = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.275384 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] use_stderr = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.275569 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.275775 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] use_syslog = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.275967 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.276151 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:52:06.276369 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.276543 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.276792 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.277059 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.277252 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.277429 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.277615 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 01 13:52:06.277782 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.278166 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.278641 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.279098 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.279508 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.279940 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.280378 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.280849 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.281248 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.281705 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.282176 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.282637 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.283044 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.283438 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.283894 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.284431 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.284867 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.285293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.285755 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.286267 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.286741 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.287217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.287647 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.288079 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.288558 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.289003 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.289455 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.289880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.290270 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.290662 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.291125 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.291557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.291990 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.292517 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.292962 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.293397 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.293792 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.294185 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.294572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.294974 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.295379 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.295812 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.296246 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.296708 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.297139 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.297616 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.298073 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.298516 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.298948 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.299333 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.299766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.300201 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.300676 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.301089 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.301514 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.301946 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.302431 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.302880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.303304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.303726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.304181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.304668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.305104 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.305555 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.306011 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.306522 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.306959 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.307541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.307876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.308351 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.308786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.309240 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.309622 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.310114 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.310468 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.310904 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.311327 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.311762 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.312227 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.312662 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.313098 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.313644 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.314065 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.314509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.314958 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.315373 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.315772 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.316193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.316644 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.317067 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.317495 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.317929 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.318346 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.318750 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.319168 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.319580 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.320039 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.320530 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.321017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.321521 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.321972 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.322414 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] auth.external = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.322833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.323331 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.324123 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.324123 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] auth.password = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.325021 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] auth.token = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.325021 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.327537 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.327537 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.327537 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.327537 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.327537 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.327537 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.329822 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] credential.caching = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.329822 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.329822 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.329822 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.329822 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.331313 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.331313 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.331313 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.331313 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.331989 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.332120 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.332661 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.333102 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.333547 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.333931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.334377 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.334764 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] federation.caching = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.335155 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.335583 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.335751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.336030 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.336228 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.336474 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.336674 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.336921 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.337354 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.337868 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.338240 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.338702 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.339137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.339540 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.339945 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity.caching = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.340426 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.340893 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.341360 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.341880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.342210 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.342626 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.343182 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.343912 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.344096 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.344471 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.344912 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.345346 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.345759 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.348389 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.348389 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.348389 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.348389 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.348389 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.348389 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.350012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.350012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.350012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.350012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.350012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.351173 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.351173 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.351572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.351962 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.352413 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.352846 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.353248 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.353779 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.353964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.354165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.354553 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.354958 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.355365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.355807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.356250 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.356732 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.357171 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.357607 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.358089 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.358548 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.358950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.359371 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.359847 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.360353 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.360851 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.361399 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.361743 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.362206 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.362615 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.363101 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.363503 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.363960 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.364498 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.364907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.365294 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.365695 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.366135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.366579 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.366976 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.367361 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.367769 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.368190 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.368656 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.369094 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.369538 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.369923 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.370349 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.370752 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.371163 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.371575 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.372003 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.372483 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.372939 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.373350 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.373814 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.374227 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.374653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.375083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.375572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.376343 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.376786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.377228 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.377665 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.378064 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.378499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] resource.caching = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.378977 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.379419 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.379868 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.380330 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.380818 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.381241 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.381685 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.382115 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.382541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.382947 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] role.caching = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.383399 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] role.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.383818 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.384318 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.384744 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.385196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.385663 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.386119 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.386512 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.386932 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.387347 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.388710 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.388710 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.388710 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.390614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.390614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.390614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.390614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.391373 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.391373 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.391781 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.392218 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.392958 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.393381 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.393798 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.394198 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.394573 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.394979 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.395379 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.395810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.396308 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.396836 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.397108 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.397497 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.397885 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.398309 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] token.caching = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.398709 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.399122 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.399525 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.399968 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.400425 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.400860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.401279 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.401680 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.402109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.402514 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.402945 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.403370 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.403822 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.404318 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.404710 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.405143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.405668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.406135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.connection = **** {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.406599 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.407077 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.407479 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.407881 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.408301 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.408690 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.409076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.409491 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.410393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.410393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.410952 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.411168 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.411347 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.411542 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.411719 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.412007 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.412467 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.412883 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.413305 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.413726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:52:06.414025 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 01 13:52:06.458475 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.458824 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.459181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.473984 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.508409 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.508774 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.512300 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.573327 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.614412 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.614825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.637664 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.638017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.739353 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.739728 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.740063 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.740456 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.740805 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.740931 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.741154 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.741316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.741707 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.742097 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.742481 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.742867 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.781841 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.782202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.782530 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.782853 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.783116 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.783269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.783488 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.783608 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.783839 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.783962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.784401 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.784745 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.785089 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.785478 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.803626 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.806889 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.825411 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.825734 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.826042 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.826378 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.826710 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.827038 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.827289 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.827396 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.827695 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.828092 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.828518 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.828989 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.829416 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.829784 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:52:06.857702 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.858022 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 13:52:06.858506 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.858790 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 13:52:06.859407 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.859555 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.860391 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.860575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.860575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 13:52:06.860847 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.861063 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.861382 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 13:52:06.861919 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.862302 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.862760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.863231 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.879007 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.879399 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.879791 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.880201 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.883477 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.883950 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.884507 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.884905 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.893013 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:06.899614 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:06.900875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:06.901346 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.901805 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.902265 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.902723 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.903148 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.903581 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.904039 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.904527 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.904988 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.905430 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.908209 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:06.908715 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.909179 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.909640 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.910109 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.910543 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.910992 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.911454 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.911907 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.912409 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.912871 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.947541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.950301 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:06.950797 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.957482 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.959677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:06.960120 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:06.960257 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.960603 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.960761 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.969922 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:06.970360 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.970826 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.972967 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:06.973411 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.980764 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:06.981239 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.981647 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.982066 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.982507 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.982957 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.983351 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.983561 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:06.983728 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.984044 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.991908 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:06.992429 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.992879 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.993330 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.993763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.994224 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.994665 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:06.995101 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.012683 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.013170 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.020612 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:07.026351 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.026487 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.026832 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.026993 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.034072 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:07.035073 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:07.040401 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.040834 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.040978 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:07.046479 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.046948 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.047367 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.047894 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.048363 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.048821 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.049457 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:07.055745 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:07.060445 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.060759 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.061043 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.061331 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.061608 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.061897 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.078357 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:07.078813 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.079226 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.080025 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.093628 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.093918 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:07.094411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.094874 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.095265 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.096847 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.097283 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.103169 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:07.103605 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.104048 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.104507 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.104933 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.107798 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.112074 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.112581 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.118833 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:07.119352 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.119870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.120428 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.120961 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.137250 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:07.138120 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.138606 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.139376 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.140048 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.140507 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.141024 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.149120 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:07.149618 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.150199 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.150637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.151043 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.151485 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.151973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.169177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:07.174204 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:07.179104 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.182344 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:07.183088 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.185650 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:07.186115 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.186495 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.187958 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:07.193679 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.194284 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.195988 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:07.198307 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.199037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.199328 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.199714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.200102 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.201474 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:07.201936 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.202397 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.209186 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:07.209702 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.210008 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.214753 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.217575 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:07.217784 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:07.217953 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.218257 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.218605 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.219055 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.219256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.219786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.220225 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.220737 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.221046 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.221443 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.221724 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.222042 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.222453 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.223466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.223971 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.224493 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.224975 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.240416 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:07.240928 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.241345 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.254657 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:52:07.255158 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.255634 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.256114 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.256625 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.257102 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.257597 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.258068 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.258730 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.259236 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.259755 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.260193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.260731 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.261243 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:52:07.262657 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: WSGI app 0 (mountpoint='') ready in 8 seconds on interpreter 0x5560fe4e79c0 pid: 80310 (default app) Aug 01 13:52:07.269940 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506fa0a-88ea-4569-84e0-f0f95ca15980 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:07.270413 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506fa0a-88ea-4569-84e0-f0f95ca15980 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:07.270637 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506fa0a-88ea-4569-84e0-f0f95ca15980 None None] PATH_INFO: `/v3` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:07.275602 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1/1] 10.222.0.45 () {50 vars in 791 bytes} [Mon Aug 1 13:52:07 2022] GET /identity/v3/ => generated 255 bytes in 12 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:07.329952 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: WSGI app 0 (mountpoint='') ready in 8 seconds on interpreter 0x5560fe4e79c0 pid: 80311 (default app) Aug 01 13:52:16.210925 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba5746-d834-48ae-bf3d-f3c472d87909 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:16.211495 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba5746-d834-48ae-bf3d-f3c472d87909 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:16.211495 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba5746-d834-48ae-bf3d-f3c472d87909 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:16.212903 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 2/2] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:52:16.221092 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede614b-984c-43b1-bf5b-02c818ee3a27 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:16.221646 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede614b-984c-43b1-bf5b-02c818ee3a27 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:16.221646 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede614b-984c-43b1-bf5b-02c818ee3a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:16.294312 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: INFO dbcounter [None req-1ede614b-984c-43b1-bf5b-02c818ee3a27 None None] Registered counter for database keystone Aug 01 13:52:16.315143 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG oslo_db.sqlalchemy.engines [None req-1ede614b-984c-43b1-bf5b-02c818ee3a27 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80311) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 01 13:52:16.322033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writer thread running {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 01 13:52:16.523712 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG passlib.handlers.bcrypt [None req-1ede614b-984c-43b1-bf5b-02c818ee3a27 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80311) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 01 13:52:16.524187 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG passlib.handlers.bcrypt [None req-1ede614b-984c-43b1-bf5b-02c818ee3a27 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80311) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 01 13:52:16.582528 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-1ede614b-984c-43b1-bf5b-02c818ee3a27 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:16.584161 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1ede614b-984c-43b1-bf5b-02c818ee3a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:16.752459 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1/3] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 535 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 01 13:52:16.892233 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba5746-d834-48ae-bf3d-f3c472d87909 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:16.964518 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: INFO dbcounter [None req-23ba5746-d834-48ae-bf3d-f3c472d87909 None None] Registered counter for database keystone Aug 01 13:52:16.989155 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG oslo_db.sqlalchemy.engines [None req-23ba5746-d834-48ae-bf3d-f3c472d87909 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80310) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 01 13:52:16.997065 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writer thread running {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 01 13:52:17.126137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba5746-d834-48ae-bf3d-f3c472d87909 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:17.130891 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766fb4e-424e-461a-9985-dd2ce620026b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:17.131624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766fb4e-424e-461a-9985-dd2ce620026b admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:17.131914 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766fb4e-424e-461a-9985-dd2ce620026b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:17.132220 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766fb4e-424e-461a-9985-dd2ce620026b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:17.142042 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8766fb4e-424e-461a-9985-dd2ce620026b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:17.195072 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8766fb4e-424e-461a-9985-dd2ce620026b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 01 13:52:17.195282 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8766fb4e-424e-461a-9985-dd2ce620026b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:17.208993 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 3/4] 10.222.0.45 () {58 vars in 1324 bytes} [Mon Aug 1 13:52:16 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 13:52:17.215327 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ede614b-984c-43b1-bf5b-02c818ee3a27 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:17.230830 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ede614b-984c-43b1-bf5b-02c818ee3a27 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:17.235920 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5fbbd9-2d67-45a9-85aa-4310c24d68c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:17.237244 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5fbbd9-2d67-45a9-85aa-4310c24d68c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:17.237580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5fbbd9-2d67-45a9-85aa-4310c24d68c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:17.237904 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5fbbd9-2d67-45a9-85aa-4310c24d68c7 admin admin] PATH_INFO: `/v3/projects/2a12613c4ac44ad788edb767368b347b` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:17.239539 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5fbbd9-2d67-45a9-85aa-4310c24d68c7 admin admin] RBAC: Authorizing `identity:get_project(project_id=2a12613c4ac44ad788edb767368b347b)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:17.291951 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8c5fbbd9-2d67-45a9-85aa-4310c24d68c7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 01 13:52:17.292316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5fbbd9-2d67-45a9-85aa-4310c24d68c7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:17.297339 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 2/5] 10.222.0.45 () {56 vars in 1207 bytes} [Mon Aug 1 13:52:17 2022] GET /identity/v3/projects/2a12613c4ac44ad788edb767368b347b => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:52:19.431369 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88efbc15-3fd3-403c-b5ff-9c7d046d6284 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:19.431922 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88efbc15-3fd3-403c-b5ff-9c7d046d6284 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:19.431922 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88efbc15-3fd3-403c-b5ff-9c7d046d6284 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:19.436121 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 4/6] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:52:19.442460 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1f9c4-c7cb-4439-9a92-496fa285d315 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:19.442992 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1f9c4-c7cb-4439-9a92-496fa285d315 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:19.443133 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1f9c4-c7cb-4439-9a92-496fa285d315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:19.469617 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-35c1f9c4-c7cb-4439-9a92-496fa285d315 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:19.471194 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-35c1f9c4-c7cb-4439-9a92-496fa285d315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:19.485657 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 3/7] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:19 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 01 13:52:19.507816 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88efbc15-3fd3-403c-b5ff-9c7d046d6284 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:19.521801 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88efbc15-3fd3-403c-b5ff-9c7d046d6284 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:19.527496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b330560-7aed-4948-9654-39810d5dc672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:19.528165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b330560-7aed-4948-9654-39810d5dc672 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:19.528488 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b330560-7aed-4948-9654-39810d5dc672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:19.528930 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b330560-7aed-4948-9654-39810d5dc672 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:19.538204 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b330560-7aed-4948-9654-39810d5dc672 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:19.540124 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b330560-7aed-4948-9654-39810d5dc672 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:19.550507 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 5/8] 10.222.0.45 () {58 vars in 1324 bytes} [Mon Aug 1 13:52:19 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 13:52:19.557589 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c1f9c4-c7cb-4439-9a92-496fa285d315 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:19.567560 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c1f9c4-c7cb-4439-9a92-496fa285d315 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:19.572912 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdb184c-29df-4d25-b726-fd2f0e41f71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:19.573573 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb184c-29df-4d25-b726-fd2f0e41f71a admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:19.573856 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb184c-29df-4d25-b726-fd2f0e41f71a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:19.574136 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb184c-29df-4d25-b726-fd2f0e41f71a admin admin] PATH_INFO: `/v3/users/bf301c0e8be34b7fa129e68f7514a379` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:19.575570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb184c-29df-4d25-b726-fd2f0e41f71a admin admin] RBAC: Authorizing `identity:get_user(user_id=bf301c0e8be34b7fa129e68f7514a379)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:19.577542 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb184c-29df-4d25-b726-fd2f0e41f71a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:19.579448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 4/9] 10.222.0.45 () {56 vars in 1198 bytes} [Mon Aug 1 13:52:19 2022] GET /identity/v3/users/bf301c0e8be34b7fa129e68f7514a379 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:21.852999 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfab480-3f99-425b-a12e-5fd82f8d13c1 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:21.853731 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfab480-3f99-425b-a12e-5fd82f8d13c1 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:21.853869 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfab480-3f99-425b-a12e-5fd82f8d13c1 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:21.856090 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 6/10] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:52:21.864627 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7ea4a-7822-4027-97c0-c3be5c28da67 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:21.865412 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be071a-1041-4635-a472-3ee754c52331 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:21.865412 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be071a-1041-4635-a472-3ee754c52331 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:21.865688 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7ea4a-7822-4027-97c0-c3be5c28da67 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:21.865836 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be071a-1041-4635-a472-3ee754c52331 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:21.865972 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7ea4a-7822-4027-97c0-c3be5c28da67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:21.866717 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 7/11] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:52:21.874617 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb35b5-f05a-40bc-894f-e25d8c4de70a None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:21.874934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb35b5-f05a-40bc-894f-e25d8c4de70a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:21.875504 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb35b5-f05a-40bc-894f-e25d8c4de70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:21.896185 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-05e7ea4a-7822-4027-97c0-c3be5c28da67 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:21.898199 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-05e7ea4a-7822-4027-97c0-c3be5c28da67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:21.959501 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG passlib.handlers.bcrypt [None req-70cb35b5-f05a-40bc-894f-e25d8c4de70a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80310) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 01 13:52:21.960424 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG passlib.handlers.bcrypt [None req-70cb35b5-f05a-40bc-894f-e25d8c4de70a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80310) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 01 13:52:22.004610 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 5/12] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:52:22.019332 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-70cb35b5-f05a-40bc-894f-e25d8c4de70a None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:22.022991 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-70cb35b5-f05a-40bc-894f-e25d8c4de70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:22.028011 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7ea4a-7822-4027-97c0-c3be5c28da67 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:22.057089 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:22.179089 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 8/13] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:52:22.199209 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7ea4a-7822-4027-97c0-c3be5c28da67 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:22.205062 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb35b5-f05a-40bc-894f-e25d8c4de70a None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:22.263962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d0105-25e7-436c-ace8-7d50d22730f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:22.265104 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d0105-25e7-436c-ace8-7d50d22730f4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:22.265613 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d0105-25e7-436c-ace8-7d50d22730f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:22.266155 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d0105-25e7-436c-ace8-7d50d22730f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:22.287648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d0105-25e7-436c-ace8-7d50d22730f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:52:22.304915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-954d0105-25e7-436c-ace8-7d50d22730f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:52:22.304915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:52:22.304915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:52:22.304915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:52:22.304915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:52:22.304915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:52:22.304915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:52:22.304915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:52:22.304915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:52:22.304915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:52:22.304915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:52:22.304915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:52:22.304915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:52:22.304915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:52:22.304915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:52:22.304915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:52:22.312603 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:52:22.312603 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:52:22.312603 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:52:22.312603 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:52:22.312603 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:52:22.312603 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:52:22.312603 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:52:22.312603 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:52:22.312603 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:52:22.312603 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:52:22.312603 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:52:22.312603 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:52:22.312603 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:52:22.312603 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:52:22.312603 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:52:22.312603 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:52:22.314704 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:52:22.314704 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:52:22.314704 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:52:22.314704 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:52:22.314704 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:52:22.314704 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:52:22.314704 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:52:22.314704 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:52:22.314704 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:52:22.314704 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:52:22.314704 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:52:22.314704 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:52:22.314704 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 6/14] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:52:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:52:22.320216 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d0105-25e7-436c-ace8-7d50d22730f4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:22.371169 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb35b5-f05a-40bc-894f-e25d8c4de70a None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:22.375629 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:22.426951 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf9f3e-ec11-495f-8778-8d6c0ba49cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:22.428524 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf9f3e-ec11-495f-8778-8d6c0ba49cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:22.429025 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf9f3e-ec11-495f-8778-8d6c0ba49cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:22.429553 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf9f3e-ec11-495f-8778-8d6c0ba49cc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:22.457565 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:22.485242 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d0105-25e7-436c-ace8-7d50d22730f4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:22.526670 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cf9f3e-ec11-495f-8778-8d6c0ba49cc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:22.528742 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cf9f3e-ec11-495f-8778-8d6c0ba49cc9 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:22.545294 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087113bf-5ecd-444c-aef5-79247532c2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:22.545942 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087113bf-5ecd-444c-aef5-79247532c2d6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:22.546222 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087113bf-5ecd-444c-aef5-79247532c2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:22.547311 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087113bf-5ecd-444c-aef5-79247532c2d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:22.548066 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087113bf-5ecd-444c-aef5-79247532c2d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:52:22.548555 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087113bf-5ecd-444c-aef5-79247532c2d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:22.550637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087113bf-5ecd-444c-aef5-79247532c2d6 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:22.566423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 7/15] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:52:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:52:22.574054 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087113bf-5ecd-444c-aef5-79247532c2d6 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:22.672340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 9/16] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:52:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 13:52:22.681897 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf9f3e-ec11-495f-8778-8d6c0ba49cc9 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:22.685746 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:22.816735 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf9f3e-ec11-495f-8778-8d6c0ba49cc9 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:22.843388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087113bf-5ecd-444c-aef5-79247532c2d6 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:22.865079 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0193118-abab-4c3f-bd68-c7a2b55fa83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:22.865962 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0193118-abab-4c3f-bd68-c7a2b55fa83d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:22.866238 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0193118-abab-4c3f-bd68-c7a2b55fa83d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:22.866573 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0193118-abab-4c3f-bd68-c7a2b55fa83d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:22.886623 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0193118-abab-4c3f-bd68-c7a2b55fa83d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:22.888154 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0193118-abab-4c3f-bd68-c7a2b55fa83d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:22.899062 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef456e-8ba1-44b5-8005-ef4d03b71006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:22.899945 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef456e-8ba1-44b5-8005-ef4d03b71006 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:22.900246 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef456e-8ba1-44b5-8005-ef4d03b71006 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:22.900624 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef456e-8ba1-44b5-8005-ef4d03b71006 None admin] PATH_INFO: `/v3/users/bf301c0e8be34b7fa129e68f7514a379` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:22.902777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ef456e-8ba1-44b5-8005-ef4d03b71006 None admin] RBAC: Authorizing `identity:get_user(user_id=bf301c0e8be34b7fa129e68f7514a379)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:22.904373 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ef456e-8ba1-44b5-8005-ef4d03b71006 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:22.906165 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 8/17] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:52:22 2022] GET /identity/v3/users/bf301c0e8be34b7fa129e68f7514a379 => generated 247 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:52:22.912158 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef456e-8ba1-44b5-8005-ef4d03b71006 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:22.915126 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-d0193118-abab-4c3f-bd68-c7a2b55fa83d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:52:22.915126 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:52:22.915126 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:52:22.915126 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:52:22.915126 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:52:22.915126 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:52:22.915126 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:52:22.915126 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:52:22.915126 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:52:22.915126 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:52:22.915126 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:52:22.915126 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:52:22.915126 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:52:22.915126 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:52:22.915126 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:52:22.916138 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:52:22.916138 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:52:22.916138 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:52:22.916138 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:52:22.916138 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:52:22.916138 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:52:22.916138 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:52:22.916138 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:52:22.916138 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:52:22.916138 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:52:22.916138 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:52:22.916138 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:52:22.916138 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:52:22.916138 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:52:22.916138 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:52:22.916138 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:52:22.916138 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:52:22.917903 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:52:22.917903 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:52:22.917903 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:52:22.917903 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:52:22.917903 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 10/18] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:52:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:52:22.925655 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0193118-abab-4c3f-bd68-c7a2b55fa83d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:22.937594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:22.992641 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:23.041978 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef456e-8ba1-44b5-8005-ef4d03b71006 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:23.075043 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0193118-abab-4c3f-bd68-c7a2b55fa83d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:23.086170 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab500c87-ee92-481f-b22e-2db9cbd1c20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:23.086841 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab500c87-ee92-481f-b22e-2db9cbd1c20b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:23.087717 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab500c87-ee92-481f-b22e-2db9cbd1c20b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:23.087717 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab500c87-ee92-481f-b22e-2db9cbd1c20b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:23.088804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab500c87-ee92-481f-b22e-2db9cbd1c20b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:23.090308 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab500c87-ee92-481f-b22e-2db9cbd1c20b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:23.091999 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 9/19] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:52:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:52:23.099017 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab500c87-ee92-481f-b22e-2db9cbd1c20b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:23.120780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c46a58-3c6a-4c52-8243-9504ad04df16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:23.121885 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c46a58-3c6a-4c52-8243-9504ad04df16 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:23.122364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c46a58-3c6a-4c52-8243-9504ad04df16 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:23.122874 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c46a58-3c6a-4c52-8243-9504ad04df16 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:23.124202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c46a58-3c6a-4c52-8243-9504ad04df16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:52:23.124787 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c46a58-3c6a-4c52-8243-9504ad04df16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:23.127578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c46a58-3c6a-4c52-8243-9504ad04df16 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:23.161273 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 11/20] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:52:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:23.220763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:23.252490 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab500c87-ee92-481f-b22e-2db9cbd1c20b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:23.308165 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ee43a-b4a9-4958-a15a-9eeaaefa9010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:23.308874 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ee43a-b4a9-4958-a15a-9eeaaefa9010 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:23.309277 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ee43a-b4a9-4958-a15a-9eeaaefa9010 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:23.309707 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ee43a-b4a9-4958-a15a-9eeaaefa9010 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:23.311104 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95ee43a-b4a9-4958-a15a-9eeaaefa9010 None admin] RBAC: Adding query filter params (role.id=3e1a9ff016c840c5b51ef7e3da9900ac, scope.domain.id=default, user.id=bf301c0e8be34b7fa129e68f7514a379) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:52:23.311613 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95ee43a-b4a9-4958-a15a-9eeaaefa9010 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:23.314262 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95ee43a-b4a9-4958-a15a-9eeaaefa9010 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:23.324171 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 10/21] 10.222.0.45 () {56 vars in 1322 bytes} [Mon Aug 1 13:52:23 2022] GET /identity/v3/role_assignments?user.id=bf301c0e8be34b7fa129e68f7514a379&scope.domain.id=default&role.id=3e1a9ff016c840c5b51ef7e3da9900ac => generated 233 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:52:26.694236 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a5a97-98c4-4932-806d-ce151e5e8835 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:26.695017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a5a97-98c4-4932-806d-ce151e5e8835 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:26.695017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a5a97-98c4-4932-806d-ce151e5e8835 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:26.696302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 12/22] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:52:26.709316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649d750-1db0-4d3b-aa35-ccc2b1731982 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:26.709838 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649d750-1db0-4d3b-aa35-ccc2b1731982 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:26.710314 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649d750-1db0-4d3b-aa35-ccc2b1731982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:26.747600 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4dcf80-86b3-4a41-bc03-eff59f50c25b None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:26.748239 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4dcf80-86b3-4a41-bc03-eff59f50c25b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:26.748729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4dcf80-86b3-4a41-bc03-eff59f50c25b None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:26.750990 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 13/23] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:52:26.752746 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-4649d750-1db0-4d3b-aa35-ccc2b1731982 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:26.755378 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4649d750-1db0-4d3b-aa35-ccc2b1731982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:26.757926 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf36266-82a5-4733-b8f1-9e3bc2d1538e None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:26.758763 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf36266-82a5-4733-b8f1-9e3bc2d1538e None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:26.759589 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf36266-82a5-4733-b8f1-9e3bc2d1538e None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:26.762031 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 14/24] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:52:26.763496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b699a3-8c11-4309-a2dc-b7f863e04211 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:26.764114 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b699a3-8c11-4309-a2dc-b7f863e04211 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:26.764597 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b699a3-8c11-4309-a2dc-b7f863e04211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:26.811597 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-63b699a3-8c11-4309-a2dc-b7f863e04211 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:26.812848 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-63b699a3-8c11-4309-a2dc-b7f863e04211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:26.960648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 11/25] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:52:26.963496 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03576f68-881f-464a-8641-163d96de986d None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:26.965837 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03576f68-881f-464a-8641-163d96de986d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:26.966628 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03576f68-881f-464a-8641-163d96de986d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:27.033458 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-03576f68-881f-464a-8641-163d96de986d None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:27.037916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-03576f68-881f-464a-8641-163d96de986d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:27.068745 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 15/26] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:52:27.070105 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b699a3-8c11-4309-a2dc-b7f863e04211 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:27.148869 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:27.226447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 12/27] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:52:27.227408 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03576f68-881f-464a-8641-163d96de986d None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:27.236945 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:27.324460 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b699a3-8c11-4309-a2dc-b7f863e04211 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:27.387112 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03576f68-881f-464a-8641-163d96de986d None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:27.453597 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc243ab-ba68-4c04-8d5f-62aefe31f188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:27.454781 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc243ab-ba68-4c04-8d5f-62aefe31f188 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:27.455262 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc243ab-ba68-4c04-8d5f-62aefe31f188 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:27.455423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc86bb-0ede-4ecc-85dc-c4acfcf5edde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:27.455743 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc243ab-ba68-4c04-8d5f-62aefe31f188 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:27.456589 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc86bb-0ede-4ecc-85dc-c4acfcf5edde None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:27.457549 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc86bb-0ede-4ecc-85dc-c4acfcf5edde None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:27.458068 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc86bb-0ede-4ecc-85dc-c4acfcf5edde None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:27.478973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adc86bb-0ede-4ecc-85dc-c4acfcf5edde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:52:27.481687 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc243ab-ba68-4c04-8d5f-62aefe31f188 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:27.484847 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc243ab-ba68-4c04-8d5f-62aefe31f188 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:27.492193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-8adc86bb-0ede-4ecc-85dc-c4acfcf5edde None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:52:27.492193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:52:27.492193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:52:27.492193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:52:27.492193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:52:27.492193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:52:27.492193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:52:27.492193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:52:27.492193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:52:27.492193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:52:27.492193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:52:27.492193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:52:27.492193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:52:27.492193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:52:27.492193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:52:27.492193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:52:27.493886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:52:27.493886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:52:27.493886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:52:27.493886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:52:27.493886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:52:27.493886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:52:27.493886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:52:27.493886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:52:27.493886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:52:27.493886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:52:27.493886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:52:27.493886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:52:27.493886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:52:27.493886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:52:27.493886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:52:27.493886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:52:27.495368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:52:27.495368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:52:27.495368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:52:27.495368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:52:27.495368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:52:27.495368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:52:27.495368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:52:27.495368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:52:27.495368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:52:27.495368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:52:27.495368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:52:27.495368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:52:27.495368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 13/28] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:52:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:52:27.496612 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc86bb-0ede-4ecc-85dc-c4acfcf5edde None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:27.519037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-5dc243ab-ba68-4c04-8d5f-62aefe31f188 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:52:27.519037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:52:27.519037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:52:27.519037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:52:27.519037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:52:27.519037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:52:27.519037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:52:27.519037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:52:27.519037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:52:27.519037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:52:27.519037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:52:27.519037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:52:27.519037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:52:27.519037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:52:27.519037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:52:27.522187 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:52:27.522187 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:52:27.522187 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:52:27.522187 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:52:27.522187 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:52:27.522187 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:52:27.522187 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:52:27.522187 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:52:27.522187 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:52:27.522187 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:52:27.522187 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:52:27.522187 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:52:27.522187 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:52:27.522187 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:52:27.522187 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:52:27.522187 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:52:27.522187 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:52:27.529801 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:52:27.529801 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:52:27.529801 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:52:27.529801 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:52:27.529801 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 16/29] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:52:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 452 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:52:27.529801 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c51ed-9275-47ad-8a34-3093bca0fd89 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:27.529801 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c51ed-9275-47ad-8a34-3093bca0fd89 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:27.529801 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c51ed-9275-47ad-8a34-3093bca0fd89 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:27.531560 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 17/30] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:27 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:52:27.536304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c51ed-9275-47ad-8a34-3093bca0fd89 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:27.664078 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:27.729578 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc86bb-0ede-4ecc-85dc-c4acfcf5edde None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:27.758914 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:27.770926 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c51ed-9275-47ad-8a34-3093bca0fd89 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:27.807706 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350279d9-d42e-4679-b589-016619ef5549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:27.809059 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350279d9-d42e-4679-b589-016619ef5549 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:27.809687 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350279d9-d42e-4679-b589-016619ef5549 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:27.810317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350279d9-d42e-4679-b589-016619ef5549 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:27.811898 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350279d9-d42e-4679-b589-016619ef5549 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:27.814861 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350279d9-d42e-4679-b589-016619ef5549 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:28.005121 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 14/31] 10.222.0.45 () {60 vars in 1131 bytes} [Mon Aug 1 13:52:27 2022] POST /identity/v3/roles => generated 225 bytes in 510 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:52:28.005980 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350279d9-d42e-4679-b589-016619ef5549 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:28.008697 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80944d24-48d0-4f7c-88e5-834142f8d91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:28.009714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80944d24-48d0-4f7c-88e5-834142f8d91a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:28.010633 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80944d24-48d0-4f7c-88e5-834142f8d91a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:28.011131 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80944d24-48d0-4f7c-88e5-834142f8d91a None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:28.020603 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80944d24-48d0-4f7c-88e5-834142f8d91a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:52:28.020603 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80944d24-48d0-4f7c-88e5-834142f8d91a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:28.020603 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80944d24-48d0-4f7c-88e5-834142f8d91a None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:28.041219 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 18/32] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:52:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 510 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:28.044208 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33efcf8-172a-4042-9298-113fce3031a7 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:28.045536 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33efcf8-172a-4042-9298-113fce3031a7 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:28.046460 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33efcf8-172a-4042-9298-113fce3031a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:28.076692 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-e33efcf8-172a-4042-9298-113fce3031a7 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:28.079926 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e33efcf8-172a-4042-9298-113fce3031a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:28.202152 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350279d9-d42e-4679-b589-016619ef5549 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:28.231562 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 19/33] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:52:28.233992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9898c-e714-4f5d-b481-70369f5fa85a None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:28.234586 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9898c-e714-4f5d-b481-70369f5fa85a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:28.235035 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9898c-e714-4f5d-b481-70369f5fa85a None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:28.243329 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 20/34] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:52:28.245115 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9898c-e714-4f5d-b481-70369f5fa85a None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:28.252826 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:28.274789 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4803cf0-ed9b-4b40-8142-63128d7851ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:28.275883 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4803cf0-ed9b-4b40-8142-63128d7851ab None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:28.276442 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4803cf0-ed9b-4b40-8142-63128d7851ab None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:28.276963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4803cf0-ed9b-4b40-8142-63128d7851ab None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:28.278483 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4803cf0-ed9b-4b40-8142-63128d7851ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:52:28.278483 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4803cf0-ed9b-4b40-8142-63128d7851ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:28.280536 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4803cf0-ed9b-4b40-8142-63128d7851ab None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:28.311300 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 15/35] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:52:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:28.315466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be48f847-a5dd-4db3-ad48-8b17ca2cafc7 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:28.315466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be48f847-a5dd-4db3-ad48-8b17ca2cafc7 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:28.316197 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be48f847-a5dd-4db3-ad48-8b17ca2cafc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:28.348458 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-be48f847-a5dd-4db3-ad48-8b17ca2cafc7 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:28.352159 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-be48f847-a5dd-4db3-ad48-8b17ca2cafc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:28.425209 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:28.448739 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9898c-e714-4f5d-b481-70369f5fa85a None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:28.458308 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 16/36] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:52:28.459251 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be48f847-a5dd-4db3-ad48-8b17ca2cafc7 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:28.506373 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a36ff5-266b-45f0-8f8c-4eacb0616a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:28.507186 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a36ff5-266b-45f0-8f8c-4eacb0616a39 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:28.507523 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a36ff5-266b-45f0-8f8c-4eacb0616a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:28.507855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a36ff5-266b-45f0-8f8c-4eacb0616a39 None admin] PATH_INFO: `/v3/users/bf301c0e8be34b7fa129e68f7514a379` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:28.511124 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a36ff5-266b-45f0-8f8c-4eacb0616a39 None admin] RBAC: Authorizing `identity:get_user(user_id=bf301c0e8be34b7fa129e68f7514a379)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:28.512458 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a36ff5-266b-45f0-8f8c-4eacb0616a39 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:28.514612 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 21/37] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:52:28 2022] GET /identity/v3/users/bf301c0e8be34b7fa129e68f7514a379 => generated 247 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:28.515484 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a36ff5-266b-45f0-8f8c-4eacb0616a39 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:28.621112 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be48f847-a5dd-4db3-ad48-8b17ca2cafc7 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:28.646392 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:28.680182 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a36ff5-266b-45f0-8f8c-4eacb0616a39 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:28.683417 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b8df1e-c64a-44f7-8a5c-1d76bf249c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:28.684459 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8df1e-c64a-44f7-8a5c-1d76bf249c0b None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:28.684990 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8df1e-c64a-44f7-8a5c-1d76bf249c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:28.685444 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8df1e-c64a-44f7-8a5c-1d76bf249c0b None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:28.686673 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b8df1e-c64a-44f7-8a5c-1d76bf249c0b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:28.689157 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b8df1e-c64a-44f7-8a5c-1d76bf249c0b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:28.713703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 17/38] 10.222.0.45 () {60 vars in 1131 bytes} [Mon Aug 1 13:52:28 2022] POST /identity/v3/roles => generated 219 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:52:28.715782 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08930dde-e30a-4e5d-bcca-ca39a7d4f2ef None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:28.716243 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08930dde-e30a-4e5d-bcca-ca39a7d4f2ef None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:28.716729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08930dde-e30a-4e5d-bcca-ca39a7d4f2ef None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:28.719078 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 18/39] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:52:28.720583 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08930dde-e30a-4e5d-bcca-ca39a7d4f2ef None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:28.751007 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a0139-bbb7-4fd8-a864-2536ffe0213b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:28.752029 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a0139-bbb7-4fd8-a864-2536ffe0213b None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:28.752545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a0139-bbb7-4fd8-a864-2536ffe0213b None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:28.752990 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a0139-bbb7-4fd8-a864-2536ffe0213b None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:28.754219 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a0139-bbb7-4fd8-a864-2536ffe0213b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:28.757021 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a0139-bbb7-4fd8-a864-2536ffe0213b None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:28.916432 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08930dde-e30a-4e5d-bcca-ca39a7d4f2ef None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:28.956306 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 22/40] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Aug 1 13:52:28 2022] POST /identity/v3/projects => generated 301 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:52:28.956306 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a0139-bbb7-4fd8-a864-2536ffe0213b None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:28.973973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb9a29c-fdd7-4303-b15d-2b3e85262f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:28.975048 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9a29c-fdd7-4303-b15d-2b3e85262f57 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:28.975507 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9a29c-fdd7-4303-b15d-2b3e85262f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:28.975953 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9a29c-fdd7-4303-b15d-2b3e85262f57 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:28.980079 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:28.980435 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb9a29c-fdd7-4303-b15d-2b3e85262f57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:28.983088 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb9a29c-fdd7-4303-b15d-2b3e85262f57 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:28.986200 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 19/41] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:52:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:28.990115 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb718d0a-795b-42bf-acea-5d33d365309f None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:28.990651 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb718d0a-795b-42bf-acea-5d33d365309f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:28.991109 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb718d0a-795b-42bf-acea-5d33d365309f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:29.026683 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-fb718d0a-795b-42bf-acea-5d33d365309f None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:29.028894 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fb718d0a-795b-42bf-acea-5d33d365309f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:29.095887 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:29.157958 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 20/42] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:52:29.159527 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb718d0a-795b-42bf-acea-5d33d365309f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:29.171845 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a0139-bbb7-4fd8-a864-2536ffe0213b None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:29.239389 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4638664-b27a-4aca-a5b3-2b5199bad41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:29.240093 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4638664-b27a-4aca-a5b3-2b5199bad41e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:29.240416 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4638664-b27a-4aca-a5b3-2b5199bad41e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:29.240705 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4638664-b27a-4aca-a5b3-2b5199bad41e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:29.243186 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4638664-b27a-4aca-a5b3-2b5199bad41e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:29.244180 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4638664-b27a-4aca-a5b3-2b5199bad41e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:29.245759 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 23/43] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:52:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:29.246546 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4638664-b27a-4aca-a5b3-2b5199bad41e None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:29.334556 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb718d0a-795b-42bf-acea-5d33d365309f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:29.351280 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:29.383521 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d81f8f-f61b-46b0-9baf-3bb1352ce317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:29.384456 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d81f8f-f61b-46b0-9baf-3bb1352ce317 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:29.384774 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d81f8f-f61b-46b0-9baf-3bb1352ce317 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:29.385080 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d81f8f-f61b-46b0-9baf-3bb1352ce317 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:29.385811 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d81f8f-f61b-46b0-9baf-3bb1352ce317 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:29.387322 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d81f8f-f61b-46b0-9baf-3bb1352ce317 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:29.387418 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4638664-b27a-4aca-a5b3-2b5199bad41e None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:29.426911 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 21/44] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Aug 1 13:52:29 2022] POST /identity/v3/projects => generated 312 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:52:29.428023 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d81f8f-f61b-46b0-9baf-3bb1352ce317 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:29.461947 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0237d-b1a3-4872-98bc-6919a4bd1e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:29.462887 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0237d-b1a3-4872-98bc-6919a4bd1e7b None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:29.463324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0237d-b1a3-4872-98bc-6919a4bd1e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:29.463745 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0237d-b1a3-4872-98bc-6919a4bd1e7b None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:29.464958 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc0237d-b1a3-4872-98bc-6919a4bd1e7b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:29.467486 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc0237d-b1a3-4872-98bc-6919a4bd1e7b None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:29.487092 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:29.503041 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 24/45] 10.222.0.45 () {60 vars in 1131 bytes} [Mon Aug 1 13:52:29 2022] POST /identity/v3/roles => generated 223 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:52:29.648704 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d81f8f-f61b-46b0-9baf-3bb1352ce317 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:29.705684 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462cf46-e7bc-41e5-8031-56ee2fc5b8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:29.706681 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462cf46-e7bc-41e5-8031-56ee2fc5b8ff None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:29.707125 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462cf46-e7bc-41e5-8031-56ee2fc5b8ff None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:29.707589 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462cf46-e7bc-41e5-8031-56ee2fc5b8ff None admin] PATH_INFO: `/v3/domains/default/users/bf301c0e8be34b7fa129e68f7514a379/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:29.710810 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8462cf46-e7bc-41e5-8031-56ee2fc5b8ff None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=bf301c0e8be34b7fa129e68f7514a379, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:29.712416 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8462cf46-e7bc-41e5-8031-56ee2fc5b8ff None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:29.733386 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 22/46] 10.222.0.45 () {58 vars in 1361 bytes} [Mon Aug 1 13:52:29 2022] PUT /identity/v3/domains/default/users/bf301c0e8be34b7fa129e68f7514a379/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:52:31.971381 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee7d206-b61e-49d4-ad04-a59210e0ea5d None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:31.972235 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee7d206-b61e-49d4-ad04-a59210e0ea5d None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:31.972235 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee7d206-b61e-49d4-ad04-a59210e0ea5d None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:31.974244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 25/47] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:52:31.981525 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74417c50-9e2e-45f4-8093-2d466284371a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:31.982328 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74417c50-9e2e-45f4-8093-2d466284371a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:31.982472 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74417c50-9e2e-45f4-8093-2d466284371a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:32.004008 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-74417c50-9e2e-45f4-8093-2d466284371a None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:32.005538 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-74417c50-9e2e-45f4-8093-2d466284371a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:32.093595 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 23/48] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:52:32.123463 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee7d206-b61e-49d4-ad04-a59210e0ea5d None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:32.312911 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee7d206-b61e-49d4-ad04-a59210e0ea5d None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:32.361433 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087d3a2-021f-43a6-8330-5231990ce67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:32.362276 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087d3a2-021f-43a6-8330-5231990ce67d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:32.362571 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087d3a2-021f-43a6-8330-5231990ce67d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:32.362898 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087d3a2-021f-43a6-8330-5231990ce67d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:32.364475 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d087d3a2-021f-43a6-8330-5231990ce67d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:32.366277 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d087d3a2-021f-43a6-8330-5231990ce67d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:32.368258 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 26/49] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:52:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:32.375273 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74417c50-9e2e-45f4-8093-2d466284371a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:32.399030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:32.433231 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429718d8-7705-44d9-8659-07ec6cbca59d None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:32.433879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429718d8-7705-44d9-8659-07ec6cbca59d None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:32.434278 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429718d8-7705-44d9-8659-07ec6cbca59d None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:32.436045 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 27/50] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:52:32.442559 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3fb19-4194-4b26-b71c-85fa8998efb7 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:32.443009 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3fb19-4194-4b26-b71c-85fa8998efb7 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:32.443637 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3fb19-4194-4b26-b71c-85fa8998efb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:32.462227 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:32.470131 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-1cf3fb19-4194-4b26-b71c-85fa8998efb7 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:32.473266 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1cf3fb19-4194-4b26-b71c-85fa8998efb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:32.553605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74417c50-9e2e-45f4-8093-2d466284371a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:32.612798 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6615f31-9417-4e96-b90c-89a39d6a5815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:32.613116 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 28/51] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:52:32.613749 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6615f31-9417-4e96-b90c-89a39d6a5815 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:32.614144 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6615f31-9417-4e96-b90c-89a39d6a5815 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:32.614554 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6615f31-9417-4e96-b90c-89a39d6a5815 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:32.615634 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6615f31-9417-4e96-b90c-89a39d6a5815 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:32.617509 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6615f31-9417-4e96-b90c-89a39d6a5815 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:32.647628 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3fb19-4194-4b26-b71c-85fa8998efb7 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:32.652607 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 24/52] 10.222.0.45 () {60 vars in 1140 bytes} [Mon Aug 1 13:52:32 2022] POST /identity/v3/projects => generated 298 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:52:32.833316 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3fb19-4194-4b26-b71c-85fa8998efb7 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:32.893628 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062a997-c2df-4ab2-a7f5-fda3299147df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:32.894639 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062a997-c2df-4ab2-a7f5-fda3299147df None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:32.894639 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062a997-c2df-4ab2-a7f5-fda3299147df None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:32.894814 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062a997-c2df-4ab2-a7f5-fda3299147df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:32.898800 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:32.910068 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1062a997-c2df-4ab2-a7f5-fda3299147df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:52:32.918012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-1062a997-c2df-4ab2-a7f5-fda3299147df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:52:32.918012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:52:32.918012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:52:32.918012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:52:32.918012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:52:32.918012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:52:32.918012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:52:32.918012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:52:32.918012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:52:32.918012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:52:32.918012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:52:32.918012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:52:32.918012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:52:32.918012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:52:32.918012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:52:32.918012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:52:32.919102 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:52:32.919102 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:52:32.919102 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:52:32.919102 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:52:32.919102 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:52:32.919102 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:52:32.919102 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:52:32.919102 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:52:32.919102 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:52:32.919102 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:52:32.919102 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:52:32.919102 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:52:32.919102 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:52:32.919102 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:52:32.919102 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:52:32.919102 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:52:32.920007 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:52:32.920007 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:52:32.920007 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:52:32.920007 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:52:32.920007 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:52:32.920007 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:52:32.920007 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:52:32.920007 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:52:32.920007 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:52:32.920007 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:52:32.920007 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:52:32.920007 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:52:32.920007 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 29/53] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:52:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:52:32.925725 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6615f31-9417-4e96-b90c-89a39d6a5815 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:33.014380 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:33.058267 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6615f31-9417-4e96-b90c-89a39d6a5815 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:33.096534 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286415f-eaa4-4542-8775-1b4dc42841a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:33.097271 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286415f-eaa4-4542-8775-1b4dc42841a5 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:33.097555 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286415f-eaa4-4542-8775-1b4dc42841a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:33.097828 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286415f-eaa4-4542-8775-1b4dc42841a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:33.098552 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b286415f-eaa4-4542-8775-1b4dc42841a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:52:33.098858 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b286415f-eaa4-4542-8775-1b4dc42841a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:33.100296 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b286415f-eaa4-4542-8775-1b4dc42841a5 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:33.119363 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 25/54] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:52:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:33.121587 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062a997-c2df-4ab2-a7f5-fda3299147df None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:33.260444 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062a997-c2df-4ab2-a7f5-fda3299147df None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:33.312982 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af3959-391c-4b7d-93fb-738161f7397d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:33.313653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af3959-391c-4b7d-93fb-738161f7397d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:33.313925 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af3959-391c-4b7d-93fb-738161f7397d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:33.314198 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af3959-391c-4b7d-93fb-738161f7397d None admin] PATH_INFO: `/v3/users/bf301c0e8be34b7fa129e68f7514a379` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:33.315434 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af3959-391c-4b7d-93fb-738161f7397d None admin] RBAC: Authorizing `identity:get_user(user_id=bf301c0e8be34b7fa129e68f7514a379)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:33.316961 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af3959-391c-4b7d-93fb-738161f7397d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:33.318618 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 30/55] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:52:33 2022] GET /identity/v3/users/bf301c0e8be34b7fa129e68f7514a379 => generated 247 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:33.324208 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286415f-eaa4-4542-8775-1b4dc42841a5 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:33.453817 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286415f-eaa4-4542-8775-1b4dc42841a5 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:33.494946 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:33.504229 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d801b8-3042-476f-bf9a-06772af8ed7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:33.504964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d801b8-3042-476f-bf9a-06772af8ed7d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:33.505419 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d801b8-3042-476f-bf9a-06772af8ed7d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:33.505738 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d801b8-3042-476f-bf9a-06772af8ed7d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:33.507048 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d801b8-3042-476f-bf9a-06772af8ed7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:33.509628 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d801b8-3042-476f-bf9a-06772af8ed7d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:33.511590 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 26/56] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:52:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:33.518018 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af3959-391c-4b7d-93fb-738161f7397d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:33.627646 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:33.678806 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af3959-391c-4b7d-93fb-738161f7397d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:33.734531 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76c939-3ada-414d-8f81-3a254ca8210d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:33.735363 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76c939-3ada-414d-8f81-3a254ca8210d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:33.735725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76c939-3ada-414d-8f81-3a254ca8210d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:33.736207 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76c939-3ada-414d-8f81-3a254ca8210d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:33.737275 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76c939-3ada-414d-8f81-3a254ca8210d None admin] RBAC: Adding query filter params (role.id=3e1a9ff016c840c5b51ef7e3da9900ac, scope.domain.id=default, user.id=bf301c0e8be34b7fa129e68f7514a379) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:52:33.737642 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76c939-3ada-414d-8f81-3a254ca8210d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:33.739169 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76c939-3ada-414d-8f81-3a254ca8210d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:33.750520 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 31/57] 10.222.0.45 () {56 vars in 1322 bytes} [Mon Aug 1 13:52:33 2022] GET /identity/v3/role_assignments?user.id=bf301c0e8be34b7fa129e68f7514a379&scope.domain.id=default&role.id=3e1a9ff016c840c5b51ef7e3da9900ac => generated 530 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:34.987286 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a12b6-0be9-49c3-9ff3-96ab156d44a5 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:34.987851 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a12b6-0be9-49c3-9ff3-96ab156d44a5 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:34.987851 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a12b6-0be9-49c3-9ff3-96ab156d44a5 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:34.989295 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 27/58] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:52:34.995058 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06e128-67cd-405d-8586-f6af5189babc None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:34.995544 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06e128-67cd-405d-8586-f6af5189babc None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:34.995629 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06e128-67cd-405d-8586-f6af5189babc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:35.015861 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-ee06e128-67cd-405d-8586-f6af5189babc None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:35.017212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ee06e128-67cd-405d-8586-f6af5189babc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:35.114643 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 32/59] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:52:35.139407 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a12b6-0be9-49c3-9ff3-96ab156d44a5 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:35.262374 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a12b6-0be9-49c3-9ff3-96ab156d44a5 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:35.299551 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8464eb0-f13a-45e9-9ca3-5177453bb990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:35.300199 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8464eb0-f13a-45e9-9ca3-5177453bb990 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:35.300506 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8464eb0-f13a-45e9-9ca3-5177453bb990 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:35.300779 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8464eb0-f13a-45e9-9ca3-5177453bb990 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:35.301938 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8464eb0-f13a-45e9-9ca3-5177453bb990 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:35.303384 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8464eb0-f13a-45e9-9ca3-5177453bb990 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:35.305051 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 28/60] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:52:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:35.310846 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06e128-67cd-405d-8586-f6af5189babc None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:35.382191 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:35.435908 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06e128-67cd-405d-8586-f6af5189babc None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:35.473221 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56279bcb-8943-48d9-b039-81a3093a87f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:35.473864 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56279bcb-8943-48d9-b039-81a3093a87f7 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:35.474145 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56279bcb-8943-48d9-b039-81a3093a87f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:35.474420 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56279bcb-8943-48d9-b039-81a3093a87f7 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:35.475132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56279bcb-8943-48d9-b039-81a3093a87f7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:35.476651 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56279bcb-8943-48d9-b039-81a3093a87f7 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:35.509555 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 33/61] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 13:52:35 2022] POST /identity/v3/users => generated 275 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:52:39.441288 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc41cf-dc37-457d-8c07-afb8f82111c5 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:39.442798 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc41cf-dc37-457d-8c07-afb8f82111c5 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:39.443539 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc41cf-dc37-457d-8c07-afb8f82111c5 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:39.446862 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 29/62] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:52:39.453145 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df6f81b-1ac1-4303-90a9-134fd463fe00 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:39.454428 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df6f81b-1ac1-4303-90a9-134fd463fe00 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:39.455117 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df6f81b-1ac1-4303-90a9-134fd463fe00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:39.489286 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-7df6f81b-1ac1-4303-90a9-134fd463fe00 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:39.491364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7df6f81b-1ac1-4303-90a9-134fd463fe00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:39.625465 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 34/63] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:52:39.674180 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc41cf-dc37-457d-8c07-afb8f82111c5 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:39.769042 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:39.774252 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a3532-0b87-4273-95ef-da92d5228320 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:39.775105 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a3532-0b87-4273-95ef-da92d5228320 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:39.775813 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a3532-0b87-4273-95ef-da92d5228320 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:39.778470 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 35/64] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:52:39.785019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fd164-e9c7-460e-8b87-c8ddd3686dfb None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:39.785635 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fd164-e9c7-460e-8b87-c8ddd3686dfb None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:39.786105 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fd164-e9c7-460e-8b87-c8ddd3686dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:39.813637 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-223fd164-e9c7-460e-8b87-c8ddd3686dfb None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:39.815721 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-223fd164-e9c7-460e-8b87-c8ddd3686dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:39.847912 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc41cf-dc37-457d-8c07-afb8f82111c5 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:39.887467 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccbe0cc-c1da-49b5-9541-6953600f46ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:39.888191 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccbe0cc-c1da-49b5-9541-6953600f46ff None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:39.888676 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccbe0cc-c1da-49b5-9541-6953600f46ff None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:39.889298 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccbe0cc-c1da-49b5-9541-6953600f46ff None admin] PATH_INFO: `/v3/roles/member` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:39.893265 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:39.904969 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccbe0cc-c1da-49b5-9541-6953600f46ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:52:39.911385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-fccbe0cc-c1da-49b5-9541-6953600f46ff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:52:39.911385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:52:39.911385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:52:39.911385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:52:39.911385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:52:39.911385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:52:39.911385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:52:39.911385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:52:39.911385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:52:39.911385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:52:39.911385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:52:39.911385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:52:39.911385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:52:39.911385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:52:39.911385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:52:39.911385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:52:39.913034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:52:39.913034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:52:39.913034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:52:39.913034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:52:39.913034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:52:39.913034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:52:39.913034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:52:39.913034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:52:39.913034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:52:39.913034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:52:39.913034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:52:39.913034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:52:39.913034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:52:39.913034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:52:39.913034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:52:39.913034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:52:39.913842 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:52:39.913842 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:52:39.913842 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:52:39.913842 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:52:39.913842 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:52:39.913842 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:52:39.913842 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:52:39.913842 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:52:39.913842 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:52:39.913842 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:52:39.913842 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:52:39.913842 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:52:39.916228 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 30/65] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:52:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:52:39.918339 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ca6f6-0948-44b8-a3b2-b8cf17849a1f None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:39.918860 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ca6f6-0948-44b8-a3b2-b8cf17849a1f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:39.919335 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ca6f6-0948-44b8-a3b2-b8cf17849a1f None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:39.920876 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 31/66] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:52:39.923210 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ca6f6-0948-44b8-a3b2-b8cf17849a1f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:39.929381 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 36/67] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:52:39.936337 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a884df-2f14-44b6-a403-b2b5427a4f10 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:39.936337 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a884df-2f14-44b6-a403-b2b5427a4f10 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:39.936337 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a884df-2f14-44b6-a403-b2b5427a4f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:39.973606 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-65a884df-2f14-44b6-a403-b2b5427a4f10 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:39.976122 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-65a884df-2f14-44b6-a403-b2b5427a4f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:40.064512 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ca6f6-0948-44b8-a3b2-b8cf17849a1f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:40.096832 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 37/68] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:52:40.098450 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a884df-2f14-44b6-a403-b2b5427a4f10 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:40.110297 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:40.124044 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9dddc-3ffd-47e1-b547-a324eb48241f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:40.124691 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9dddc-3ffd-47e1-b547-a324eb48241f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:40.125134 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9dddc-3ffd-47e1-b547-a324eb48241f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:40.125507 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9dddc-3ffd-47e1-b547-a324eb48241f None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:40.126238 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc9dddc-3ffd-47e1-b547-a324eb48241f None admin] RBAC: Adding query filter params (name=member) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:52:40.126544 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc9dddc-3ffd-47e1-b547-a324eb48241f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:40.127979 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc9dddc-3ffd-47e1-b547-a324eb48241f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:40.137350 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 32/69] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 13:52:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:52:40.138613 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990391fc-8560-49d3-85a6-6966e7c108dd None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:40.138890 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990391fc-8560-49d3-85a6-6966e7c108dd None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:40.139154 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990391fc-8560-49d3-85a6-6966e7c108dd None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:40.140684 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 33/70] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:52:40.145699 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990391fc-8560-49d3-85a6-6966e7c108dd None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:40.264031 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a884df-2f14-44b6-a403-b2b5427a4f10 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:40.269087 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:40.287841 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990391fc-8560-49d3-85a6-6966e7c108dd None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:40.316007 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209514bb-ff88-4b6e-9576-fb2addcfb30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:40.317375 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209514bb-ff88-4b6e-9576-fb2addcfb30f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:40.317867 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209514bb-ff88-4b6e-9576-fb2addcfb30f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:40.318360 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209514bb-ff88-4b6e-9576-fb2addcfb30f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:40.336434 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209514bb-ff88-4b6e-9576-fb2addcfb30f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:52:40.344660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-209514bb-ff88-4b6e-9576-fb2addcfb30f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:52:40.344660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:52:40.344660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:52:40.344660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:52:40.344660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:52:40.344660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:52:40.344660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:52:40.344660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:52:40.344660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:52:40.344660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:52:40.344660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:52:40.344660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:52:40.344660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:52:40.344660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:52:40.344660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:52:40.344660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:52:40.345659 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:52:40.345659 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:52:40.345659 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:52:40.345659 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:52:40.345659 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:52:40.345659 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:52:40.345659 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:52:40.345659 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:52:40.345659 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:52:40.345659 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:52:40.345659 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:52:40.345659 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:52:40.345659 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:52:40.345659 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:52:40.345659 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:52:40.345659 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:52:40.346458 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:52:40.346458 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:52:40.346458 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:52:40.346458 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:52:40.346458 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:52:40.346458 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:52:40.346458 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:52:40.346458 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:52:40.346458 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:52:40.346458 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:52:40.346458 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:52:40.346458 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:52:40.347102 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf06ac-744d-4f50-b259-c00f8072249a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:40.347194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 38/71] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:52:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:52:40.347992 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf06ac-744d-4f50-b259-c00f8072249a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:40.348113 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209514bb-ff88-4b6e-9576-fb2addcfb30f None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:40.348194 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf06ac-744d-4f50-b259-c00f8072249a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:40.348637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf06ac-744d-4f50-b259-c00f8072249a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:40.362006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf06ac-744d-4f50-b259-c00f8072249a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:52:40.367611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-0caf06ac-744d-4f50-b259-c00f8072249a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:52:40.367611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:52:40.367611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:52:40.367611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:52:40.367611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:52:40.367611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:52:40.367611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:52:40.367611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:52:40.367611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:52:40.367611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:52:40.367611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:52:40.367611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:52:40.367611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:52:40.367611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:52:40.367611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:52:40.367611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:52:40.368594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:52:40.368594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:52:40.368594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:52:40.368594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:52:40.368594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:52:40.368594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:52:40.368594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:52:40.368594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:52:40.368594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:52:40.368594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:52:40.368594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:52:40.368594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:52:40.368594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:52:40.368594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:52:40.368594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:52:40.368594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:52:40.369420 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:52:40.369420 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:52:40.369420 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:52:40.369420 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:52:40.369420 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:52:40.369420 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:52:40.369420 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:52:40.369420 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:52:40.369420 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:52:40.369420 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:52:40.369420 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:52:40.369420 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:52:40.369420 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 34/72] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:52:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:52:40.370125 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af5f73-7674-4853-8455-073dcd468e98 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:40.370567 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af5f73-7674-4853-8455-073dcd468e98 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:40.370696 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af5f73-7674-4853-8455-073dcd468e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:40.394365 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-06af5f73-7674-4853-8455-073dcd468e98 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:40.396018 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-06af5f73-7674-4853-8455-073dcd468e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:40.463438 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:40.483895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209514bb-ff88-4b6e-9576-fb2addcfb30f None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:40.508308 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 35/73] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:52:40.508308 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f59368-0aed-43b2-b533-65d0cc8a532a None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:40.508308 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f59368-0aed-43b2-b533-65d0cc8a532a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:40.508308 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f59368-0aed-43b2-b533-65d0cc8a532a None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:40.508308 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 36/74] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:52:40.508308 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f59368-0aed-43b2-b533-65d0cc8a532a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:40.532885 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776cf943-261d-49b0-af54-20ae1e044d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:40.533564 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776cf943-261d-49b0-af54-20ae1e044d33 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:40.533947 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776cf943-261d-49b0-af54-20ae1e044d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:40.534225 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776cf943-261d-49b0-af54-20ae1e044d33 None admin] PATH_INFO: `/v3/users/0cd5be8f747c4089988de317f6307400` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:40.561716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776cf943-261d-49b0-af54-20ae1e044d33 None admin] RBAC: Authorizing `identity:get_user(user_id=0cd5be8f747c4089988de317f6307400)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:40.563349 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776cf943-261d-49b0-af54-20ae1e044d33 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:40.565232 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 39/75] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:52:40 2022] GET /identity/v3/users/0cd5be8f747c4089988de317f6307400 => generated 275 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:40.566168 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776cf943-261d-49b0-af54-20ae1e044d33 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:40.607946 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:40.670839 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f59368-0aed-43b2-b533-65d0cc8a532a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:40.709190 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776cf943-261d-49b0-af54-20ae1e044d33 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:40.712743 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e58a29-c8fb-4310-8148-0be565247c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:40.713393 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e58a29-c8fb-4310-8148-0be565247c93 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:40.713813 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e58a29-c8fb-4310-8148-0be565247c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:40.714195 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e58a29-c8fb-4310-8148-0be565247c93 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:40.715033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e58a29-c8fb-4310-8148-0be565247c93 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:52:40.715541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e58a29-c8fb-4310-8148-0be565247c93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:40.717830 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e58a29-c8fb-4310-8148-0be565247c93 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:40.732112 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 37/76] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:52:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:40.733866 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6507c-a75e-4e1a-a9e8-8db785c2d790 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:40.734168 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6507c-a75e-4e1a-a9e8-8db785c2d790 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:40.734572 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6507c-a75e-4e1a-a9e8-8db785c2d790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:40.758338 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74747de1-6c81-436e-81e5-32adffc47bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:40.758987 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74747de1-6c81-436e-81e5-32adffc47bae None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:40.759273 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74747de1-6c81-436e-81e5-32adffc47bae None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:40.759553 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74747de1-6c81-436e-81e5-32adffc47bae None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:40.759650 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-7aa6507c-a75e-4e1a-a9e8-8db785c2d790 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:40.760314 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74747de1-6c81-436e-81e5-32adffc47bae None admin] RBAC: Adding query filter params (name=member) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:52:40.760625 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74747de1-6c81-436e-81e5-32adffc47bae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:40.760805 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7aa6507c-a75e-4e1a-a9e8-8db785c2d790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:40.762390 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74747de1-6c81-436e-81e5-32adffc47bae None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:40.777613 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 40/77] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 13:52:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:40.778630 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74747de1-6c81-436e-81e5-32adffc47bae None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:40.807089 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:40.857552 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 38/78] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:52:40.858332 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa6507c-a75e-4e1a-a9e8-8db785c2d790 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:40.909257 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:40.931373 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74747de1-6c81-436e-81e5-32adffc47bae None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:40.974252 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34000e2-8120-45f4-81b4-9a5368eb3e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:40.974922 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34000e2-8120-45f4-81b4-9a5368eb3e27 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:40.975209 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34000e2-8120-45f4-81b4-9a5368eb3e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:40.975487 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34000e2-8120-45f4-81b4-9a5368eb3e27 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:40.983510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa6507c-a75e-4e1a-a9e8-8db785c2d790 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:40.988152 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34000e2-8120-45f4-81b4-9a5368eb3e27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:52:40.993901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-f34000e2-8120-45f4-81b4-9a5368eb3e27 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:52:40.993901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:52:40.993901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:52:40.993901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:52:40.993901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:52:40.993901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:52:40.993901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:52:40.993901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:52:40.993901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:52:40.993901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:52:40.993901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:52:40.993901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:52:40.993901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:52:40.993901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:52:40.993901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:52:40.993901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:52:40.994805 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:52:40.994805 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:52:40.994805 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:52:40.994805 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:52:40.994805 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:52:40.994805 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:52:40.994805 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:52:40.994805 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:52:40.994805 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:52:40.994805 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:52:40.994805 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:52:40.994805 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:52:40.994805 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:52:40.994805 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:52:40.994805 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:52:40.994805 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:52:40.995729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:52:40.995729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:52:40.995729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:52:40.995729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:52:40.995729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:52:40.995729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:52:40.995729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:52:40.995729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:52:40.995729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:52:40.995729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:52:40.995729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:52:40.995729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:52:40.995729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 41/79] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 13:52:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:52:40.996544 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34000e2-8120-45f4-81b4-9a5368eb3e27 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:41.034169 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d53e0a-c765-475c-b8b1-6bc9315e03de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:41.035084 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d53e0a-c765-475c-b8b1-6bc9315e03de None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:41.035466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d53e0a-c765-475c-b8b1-6bc9315e03de None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:41.035899 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d53e0a-c765-475c-b8b1-6bc9315e03de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:41.078017 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:41.124578 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d53e0a-c765-475c-b8b1-6bc9315e03de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:41.125973 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34000e2-8120-45f4-81b4-9a5368eb3e27 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:41.126087 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d53e0a-c765-475c-b8b1-6bc9315e03de None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:41.168152 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe4287e-ec46-4243-b10a-6edd4eddc194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:41.168865 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4287e-ec46-4243-b10a-6edd4eddc194 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:41.169154 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4287e-ec46-4243-b10a-6edd4eddc194 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:41.169436 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4287e-ec46-4243-b10a-6edd4eddc194 None admin] PATH_INFO: `/v3/users/bf301c0e8be34b7fa129e68f7514a379` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:41.170700 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe4287e-ec46-4243-b10a-6edd4eddc194 None admin] RBAC: Authorizing `identity:get_user(user_id=bf301c0e8be34b7fa129e68f7514a379)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:41.172247 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe4287e-ec46-4243-b10a-6edd4eddc194 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:41.173985 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 42/80] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:52:40 2022] GET /identity/v3/users/bf301c0e8be34b7fa129e68f7514a379 => generated 247 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:41.174781 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe4287e-ec46-4243-b10a-6edd4eddc194 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:41.199636 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:41.276938 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 39/81] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:52:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:52:41.276938 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d53e0a-c765-475c-b8b1-6bc9315e03de None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:41.309948 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe4287e-ec46-4243-b10a-6edd4eddc194 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:41.355839 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb564e1-62a3-488b-9c03-ba623fb684f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:41.356606 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb564e1-62a3-488b-9c03-ba623fb684f2 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:41.356894 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb564e1-62a3-488b-9c03-ba623fb684f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:41.357255 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb564e1-62a3-488b-9c03-ba623fb684f2 None admin] PATH_INFO: `/v3/users/0cd5be8f747c4089988de317f6307400` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:41.357393 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:41.358903 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb564e1-62a3-488b-9c03-ba623fb684f2 None admin] RBAC: Authorizing `identity:get_user(user_id=0cd5be8f747c4089988de317f6307400)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:41.360813 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb564e1-62a3-488b-9c03-ba623fb684f2 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:41.362542 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 43/82] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:52:41 2022] GET /identity/v3/users/0cd5be8f747c4089988de317f6307400 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:41.363343 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb564e1-62a3-488b-9c03-ba623fb684f2 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:41.407844 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d53e0a-c765-475c-b8b1-6bc9315e03de None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:41.478970 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:41.482515 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165a362-55ab-4706-9780-5022798c1cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:41.483524 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165a362-55ab-4706-9780-5022798c1cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:41.483975 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165a362-55ab-4706-9780-5022798c1cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:41.484555 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165a362-55ab-4706-9780-5022798c1cb7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:41.487482 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a165a362-55ab-4706-9780-5022798c1cb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:41.489005 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a165a362-55ab-4706-9780-5022798c1cb7 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:41.491853 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 40/83] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:52:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:41.493050 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165a362-55ab-4706-9780-5022798c1cb7 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:41.512003 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb564e1-62a3-488b-9c03-ba623fb684f2 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:41.566797 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30cf62-7af0-4a87-8d90-11d6de5d1f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:41.567495 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30cf62-7af0-4a87-8d90-11d6de5d1f97 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:41.567926 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30cf62-7af0-4a87-8d90-11d6de5d1f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:41.568391 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30cf62-7af0-4a87-8d90-11d6de5d1f97 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:41.569383 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af30cf62-7af0-4a87-8d90-11d6de5d1f97 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:52:41.569726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af30cf62-7af0-4a87-8d90-11d6de5d1f97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:41.571919 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af30cf62-7af0-4a87-8d90-11d6de5d1f97 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:41.583413 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 44/84] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:52:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:41.584485 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30cf62-7af0-4a87-8d90-11d6de5d1f97 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:41.623546 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165a362-55ab-4706-9780-5022798c1cb7 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:41.669014 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb92c5-f750-4b17-9462-98e2e75faf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:41.669851 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb92c5-f750-4b17-9462-98e2e75faf98 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:41.670150 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb92c5-f750-4b17-9462-98e2e75faf98 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:41.670590 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb92c5-f750-4b17-9462-98e2e75faf98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:41.675635 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:41.720503 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30cf62-7af0-4a87-8d90-11d6de5d1f97 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:41.761422 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad89d35-2b02-4b56-b6c6-4a95588f3758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:41.761995 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cb92c5-f750-4b17-9462-98e2e75faf98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:41.762526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad89d35-2b02-4b56-b6c6-4a95588f3758 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:41.763002 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad89d35-2b02-4b56-b6c6-4a95588f3758 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:41.763384 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad89d35-2b02-4b56-b6c6-4a95588f3758 None admin] PATH_INFO: `/v3/projects/c18388255f054822b773ec88708292da` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:41.763549 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cb92c5-f750-4b17-9462-98e2e75faf98 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:41.764976 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad89d35-2b02-4b56-b6c6-4a95588f3758 None admin] RBAC: Authorizing `identity:get_project(project_id=c18388255f054822b773ec88708292da)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:41.766572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad89d35-2b02-4b56-b6c6-4a95588f3758 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:41.769088 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 45/85] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:52:41 2022] GET /identity/v3/projects/c18388255f054822b773ec88708292da => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:41.769934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad89d35-2b02-4b56-b6c6-4a95588f3758 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:41.789786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:41.880854 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 41/86] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:52:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:52:41.881946 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb92c5-f750-4b17-9462-98e2e75faf98 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:41.905901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad89d35-2b02-4b56-b6c6-4a95588f3758 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:41.925516 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:41.948971 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172a986-8483-4bbb-ba29-bd18546f855f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:41.950010 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172a986-8483-4bbb-ba29-bd18546f855f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:41.950355 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172a986-8483-4bbb-ba29-bd18546f855f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:41.950742 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172a986-8483-4bbb-ba29-bd18546f855f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:42.011193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb92c5-f750-4b17-9462-98e2e75faf98 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:42.035604 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172a986-8483-4bbb-ba29-bd18546f855f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:42.037269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172a986-8483-4bbb-ba29-bd18546f855f None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:42.052003 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:42.053666 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5ac20-5f89-45a5-9a9b-132b6cccb121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:42.054503 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5ac20-5f89-45a5-9a9b-132b6cccb121 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:42.054850 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5ac20-5f89-45a5-9a9b-132b6cccb121 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:42.055197 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5ac20-5f89-45a5-9a9b-132b6cccb121 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:42.056023 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5ac20-5f89-45a5-9a9b-132b6cccb121 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:42.057808 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5ac20-5f89-45a5-9a9b-132b6cccb121 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:42.092031 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 42/87] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 13:52:41 2022] POST /identity/v3/users => generated 289 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:52:42.093870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5ac20-5f89-45a5-9a9b-132b6cccb121 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:42.183602 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 46/88] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:52:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:52:42.184998 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172a986-8483-4bbb-ba29-bd18546f855f None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:42.243414 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:42.261984 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5ac20-5f89-45a5-9a9b-132b6cccb121 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:42.320772 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73e3e5-83cd-4189-a5c3-945fa7e0c22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:42.321487 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73e3e5-83cd-4189-a5c3-945fa7e0c22c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:42.321921 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73e3e5-83cd-4189-a5c3-945fa7e0c22c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:42.322204 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73e3e5-83cd-4189-a5c3-945fa7e0c22c None admin] PATH_INFO: `/v3/users/0cd5be8f747c4089988de317f6307400` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:42.323738 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73e3e5-83cd-4189-a5c3-945fa7e0c22c None admin] RBAC: Authorizing `identity:get_user(user_id=0cd5be8f747c4089988de317f6307400)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:42.326631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73e3e5-83cd-4189-a5c3-945fa7e0c22c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:42.327538 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 43/89] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:52:42 2022] GET /identity/v3/users/0cd5be8f747c4089988de317f6307400 => generated 275 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:42.328464 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73e3e5-83cd-4189-a5c3-945fa7e0c22c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:42.369333 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172a986-8483-4bbb-ba29-bd18546f855f None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:42.387333 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:42.423195 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25647b91-ca98-4d98-b964-10a6aa9744d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:42.423844 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25647b91-ca98-4d98-b964-10a6aa9744d6 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:42.424178 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25647b91-ca98-4d98-b964-10a6aa9744d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:42.424495 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25647b91-ca98-4d98-b964-10a6aa9744d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:42.426124 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25647b91-ca98-4d98-b964-10a6aa9744d6 None admin] RBAC: Adding query filter params (role.id=75aff8cb4cf748a2adbd6c3f141fe8ac, scope.project.id=c18388255f054822b773ec88708292da, user.id=0cd5be8f747c4089988de317f6307400) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:52:42.426316 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25647b91-ca98-4d98-b964-10a6aa9744d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:42.427955 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25647b91-ca98-4d98-b964-10a6aa9744d6 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:42.435794 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 47/90] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:52:42 2022] GET /identity/v3/role_assignments?user.id=0cd5be8f747c4089988de317f6307400&scope.project.id=c18388255f054822b773ec88708292da&role.id=75aff8cb4cf748a2adbd6c3f141fe8ac => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:42.436677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25647b91-ca98-4d98-b964-10a6aa9744d6 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:42.505521 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73e3e5-83cd-4189-a5c3-945fa7e0c22c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:42.576169 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b8480-8964-4135-9bd1-33f05c0593ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:42.577325 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b8480-8964-4135-9bd1-33f05c0593ce None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:42.577820 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b8480-8964-4135-9bd1-33f05c0593ce None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:42.578325 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b8480-8964-4135-9bd1-33f05c0593ce None admin] PATH_INFO: `/v3/projects/c18388255f054822b773ec88708292da` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:42.580545 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b8480-8964-4135-9bd1-33f05c0593ce None admin] RBAC: Authorizing `identity:get_project(project_id=c18388255f054822b773ec88708292da)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:42.583678 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b8480-8964-4135-9bd1-33f05c0593ce None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:42.587021 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 44/91] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:52:42 2022] GET /identity/v3/projects/c18388255f054822b773ec88708292da => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:42.588010 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b8480-8964-4135-9bd1-33f05c0593ce None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:42.628906 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25647b91-ca98-4d98-b964-10a6aa9744d6 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:42.630775 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:42.689312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040aac20-c5f6-45be-ba2e-fac9b25c5c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:42.690399 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040aac20-c5f6-45be-ba2e-fac9b25c5c9b None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:42.690902 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040aac20-c5f6-45be-ba2e-fac9b25c5c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:42.691372 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040aac20-c5f6-45be-ba2e-fac9b25c5c9b None admin] PATH_INFO: `/v3/projects/fd7418cfcedb4164bed000f138751888` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:42.692876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040aac20-c5f6-45be-ba2e-fac9b25c5c9b None admin] RBAC: Authorizing `identity:get_project(project_id=fd7418cfcedb4164bed000f138751888)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:42.694797 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040aac20-c5f6-45be-ba2e-fac9b25c5c9b None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:42.697305 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 48/92] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:52:42 2022] GET /identity/v3/projects/fd7418cfcedb4164bed000f138751888 => generated 312 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:42.698514 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040aac20-c5f6-45be-ba2e-fac9b25c5c9b None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:42.737526 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b8480-8964-4135-9bd1-33f05c0593ce None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:42.760893 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:42.817283 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1187fd07-5f46-4b0f-9be7-65f8cfef4312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:42.818825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187fd07-5f46-4b0f-9be7-65f8cfef4312 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:42.819655 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187fd07-5f46-4b0f-9be7-65f8cfef4312 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:42.820577 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187fd07-5f46-4b0f-9be7-65f8cfef4312 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:42.869909 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040aac20-c5f6-45be-ba2e-fac9b25c5c9b None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:42.940232 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ad8ae-e703-4050-860e-6ff6832504e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:42.940810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ad8ae-e703-4050-860e-6ff6832504e9 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:42.941144 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ad8ae-e703-4050-860e-6ff6832504e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:42.941427 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ad8ae-e703-4050-860e-6ff6832504e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:42.942279 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ad8ae-e703-4050-860e-6ff6832504e9 None admin] RBAC: Adding query filter params (role.id=3e1a9ff016c840c5b51ef7e3da9900ac, scope.project.id=c18388255f054822b773ec88708292da, user.id=bf301c0e8be34b7fa129e68f7514a379) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:52:42.942588 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ad8ae-e703-4050-860e-6ff6832504e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:42.944244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ad8ae-e703-4050-860e-6ff6832504e9 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:42.952055 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 49/93] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:52:42 2022] GET /identity/v3/role_assignments?user.id=bf301c0e8be34b7fa129e68f7514a379&scope.project.id=c18388255f054822b773ec88708292da&role.id=3e1a9ff016c840c5b51ef7e3da9900ac => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:42.953205 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ad8ae-e703-4050-860e-6ff6832504e9 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:43.001294 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1187fd07-5f46-4b0f-9be7-65f8cfef4312 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:43.002937 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1187fd07-5f46-4b0f-9be7-65f8cfef4312 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:43.091154 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:43.243565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ad8ae-e703-4050-860e-6ff6832504e9 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:43.259135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:43.274381 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 45/94] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:52:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 687 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:52:43.280621 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1187fd07-5f46-4b0f-9be7-65f8cfef4312 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:43.365479 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222eecb-e6d1-4af9-ae43-53e5917d50a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:43.366135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222eecb-e6d1-4af9-ae43-53e5917d50a0 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:43.366422 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222eecb-e6d1-4af9-ae43-53e5917d50a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:43.366939 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222eecb-e6d1-4af9-ae43-53e5917d50a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:43.367788 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1222eecb-e6d1-4af9-ae43-53e5917d50a0 None admin] RBAC: Adding query filter params (role.id=75aff8cb4cf748a2adbd6c3f141fe8ac, scope.project.id=fd7418cfcedb4164bed000f138751888, user.id=0cd5be8f747c4089988de317f6307400) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:52:43.368104 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1222eecb-e6d1-4af9-ae43-53e5917d50a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:43.369779 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1222eecb-e6d1-4af9-ae43-53e5917d50a0 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:43.386300 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 50/95] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:52:42 2022] GET /identity/v3/role_assignments?user.id=0cd5be8f747c4089988de317f6307400&scope.project.id=fd7418cfcedb4164bed000f138751888&role.id=75aff8cb4cf748a2adbd6c3f141fe8ac => generated 259 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:43.434997 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1187fd07-5f46-4b0f-9be7-65f8cfef4312 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:43.481095 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:43.493680 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f96e6-1973-47d6-ac20-f7c40afa486d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:43.494309 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f96e6-1973-47d6-ac20-f7c40afa486d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:43.494605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f96e6-1973-47d6-ac20-f7c40afa486d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:43.494888 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f96e6-1973-47d6-ac20-f7c40afa486d None admin] PATH_INFO: `/v3/projects/c18388255f054822b773ec88708292da` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:43.501372 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f96e6-1973-47d6-ac20-f7c40afa486d None admin] RBAC: Authorizing `identity:get_project(project_id=c18388255f054822b773ec88708292da)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:43.502950 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f96e6-1973-47d6-ac20-f7c40afa486d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:43.505112 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 46/96] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:52:43 2022] GET /identity/v3/projects/c18388255f054822b773ec88708292da => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:43.511292 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222eecb-e6d1-4af9-ae43-53e5917d50a0 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:43.648004 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222eecb-e6d1-4af9-ae43-53e5917d50a0 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:43.690756 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d938cee2-862a-4646-8f32-4ae5cc8d9aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:43.691404 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938cee2-862a-4646-8f32-4ae5cc8d9aae None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:43.691692 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938cee2-862a-4646-8f32-4ae5cc8d9aae None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:43.691969 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938cee2-862a-4646-8f32-4ae5cc8d9aae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:43.692820 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d938cee2-862a-4646-8f32-4ae5cc8d9aae None admin] RBAC: Adding query filter params (role.id=60efb70fa7354ebfb04aa5e9bdad2a78, scope.project.id=c18388255f054822b773ec88708292da, user.id=0cd5be8f747c4089988de317f6307400) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:52:43.693138 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d938cee2-862a-4646-8f32-4ae5cc8d9aae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:43.694777 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d938cee2-862a-4646-8f32-4ae5cc8d9aae None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:43.701761 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 51/97] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:52:43 2022] GET /identity/v3/role_assignments?user.id=0cd5be8f747c4089988de317f6307400&scope.project.id=c18388255f054822b773ec88708292da&role.id=60efb70fa7354ebfb04aa5e9bdad2a78 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:52:44.990524 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3d720-0293-4446-9201-603b7842c440 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:44.991056 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3d720-0293-4446-9201-603b7842c440 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:44.991167 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3d720-0293-4446-9201-603b7842c440 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:44.992859 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 47/98] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:52:44.998710 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1318b74-18b9-4305-894d-c31b7bceebe0 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:44.999243 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1318b74-18b9-4305-894d-c31b7bceebe0 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:44.999243 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1318b74-18b9-4305-894d-c31b7bceebe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:45.025476 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-b1318b74-18b9-4305-894d-c31b7bceebe0 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:45.026729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b1318b74-18b9-4305-894d-c31b7bceebe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:45.068413 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:45.126159 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 52/99] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:52:45.164493 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3d720-0293-4446-9201-603b7842c440 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:45.394775 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3d720-0293-4446-9201-603b7842c440 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:45.448228 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93f79f-9593-4fae-8eea-3654c58ffe48 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:45.448612 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93f79f-9593-4fae-8eea-3654c58ffe48 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:45.448823 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93f79f-9593-4fae-8eea-3654c58ffe48 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:45.450822 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 53/100] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:52:45.462476 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39ee93-f488-4021-98b6-ef04f786c2f6 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:45.462875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39ee93-f488-4021-98b6-ef04f786c2f6 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:45.463147 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39ee93-f488-4021-98b6-ef04f786c2f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:45.487069 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437fedf0-3e5b-4c1c-92ec-21062b5cb2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:45.488189 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fedf0-3e5b-4c1c-92ec-21062b5cb2fb None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:45.488725 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fedf0-3e5b-4c1c-92ec-21062b5cb2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:45.489227 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fedf0-3e5b-4c1c-92ec-21062b5cb2fb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:45.493519 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-0c39ee93-f488-4021-98b6-ef04f786c2f6 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:45.494708 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0c39ee93-f488-4021-98b6-ef04f786c2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:45.549450 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437fedf0-3e5b-4c1c-92ec-21062b5cb2fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:52:45.560025 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-437fedf0-3e5b-4c1c-92ec-21062b5cb2fb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:52:45.560025 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:52:45.560025 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:52:45.560025 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:52:45.560025 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:52:45.560025 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:52:45.560025 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:52:45.560025 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:52:45.560025 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:52:45.560025 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:52:45.560025 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:52:45.560025 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:52:45.560025 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:52:45.560025 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:52:45.560025 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:52:45.560025 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:52:45.561258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:52:45.561258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:52:45.561258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:52:45.561258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:52:45.561258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:52:45.561258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:52:45.561258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:52:45.561258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:52:45.561258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:52:45.561258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:52:45.561258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:52:45.561258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:52:45.561258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:52:45.561258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:52:45.561258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:52:45.561258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:52:45.562366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:52:45.562366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:52:45.562366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:52:45.562366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:52:45.562366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:52:45.562366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:52:45.562366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:52:45.562366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:52:45.562366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:52:45.562366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:52:45.562366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:52:45.562366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:52:45.570586 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 48/101] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:52:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 407 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:52:45.578005 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437fedf0-3e5b-4c1c-92ec-21062b5cb2fb None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:45.681731 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 54/102] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:52:45.696249 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:45.729672 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39ee93-f488-4021-98b6-ef04f786c2f6 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:45.756628 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437fedf0-3e5b-4c1c-92ec-21062b5cb2fb None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:45.811598 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1303ce-a708-4ecd-bf45-a83f235a820b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:45.812447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1303ce-a708-4ecd-bf45-a83f235a820b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:45.812651 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1303ce-a708-4ecd-bf45-a83f235a820b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:45.812886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1303ce-a708-4ecd-bf45-a83f235a820b None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:45.813869 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1303ce-a708-4ecd-bf45-a83f235a820b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:52:45.814223 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1303ce-a708-4ecd-bf45-a83f235a820b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:45.816065 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1303ce-a708-4ecd-bf45-a83f235a820b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:45.838259 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 49/103] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 13:52:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:45.839550 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1303ce-a708-4ecd-bf45-a83f235a820b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:46.021696 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1303ce-a708-4ecd-bf45-a83f235a820b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:46.072733 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:46.085952 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b26261-0dd7-4dc8-99d7-7a19630d64e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:46.086632 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b26261-0dd7-4dc8-99d7-7a19630d64e4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:46.086913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b26261-0dd7-4dc8-99d7-7a19630d64e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:46.087443 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b26261-0dd7-4dc8-99d7-7a19630d64e4 None admin] PATH_INFO: `/v3/users/70ba8bcd60de4443b95ce1e8d22ff2b4` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:46.118316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b26261-0dd7-4dc8-99d7-7a19630d64e4 None admin] RBAC: Authorizing `identity:get_user(user_id=70ba8bcd60de4443b95ce1e8d22ff2b4)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:46.118853 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:46.119944 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b26261-0dd7-4dc8-99d7-7a19630d64e4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:46.126267 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 50/104] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:52:45 2022] GET /identity/v3/users/70ba8bcd60de4443b95ce1e8d22ff2b4 => generated 289 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:52:46.134777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b2915-0ede-4645-9734-94926fe46223 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:46.134963 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39ee93-f488-4021-98b6-ef04f786c2f6 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:46.135290 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b2915-0ede-4645-9734-94926fe46223 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:46.135790 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b2915-0ede-4645-9734-94926fe46223 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:46.138707 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 51/105] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:46 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:52:46.145326 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6b2915-0ede-4645-9734-94926fe46223 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:46.189242 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237e541-cd54-41e5-bebc-3b6a289a6404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:46.190289 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237e541-cd54-41e5-bebc-3b6a289a6404 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:46.190740 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237e541-cd54-41e5-bebc-3b6a289a6404 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:46.191174 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237e541-cd54-41e5-bebc-3b6a289a6404 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:46.208247 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c237e541-cd54-41e5-bebc-3b6a289a6404 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:52:46.217692 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-c237e541-cd54-41e5-bebc-3b6a289a6404 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:52:46.217692 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:52:46.217692 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:52:46.217692 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:52:46.217692 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:52:46.217692 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:52:46.217692 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:52:46.217692 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:52:46.217692 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:52:46.217692 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:52:46.217692 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:52:46.217692 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:52:46.217692 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:52:46.217692 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:52:46.217692 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:52:46.217692 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:52:46.218896 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:52:46.218896 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:52:46.218896 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:52:46.218896 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:52:46.218896 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:52:46.218896 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:52:46.218896 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:52:46.218896 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:52:46.218896 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:52:46.218896 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:52:46.218896 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:52:46.218896 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:52:46.218896 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:52:46.218896 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:52:46.218896 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:52:46.218896 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:52:46.220035 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:52:46.220035 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:52:46.220035 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:52:46.220035 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:52:46.220035 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:52:46.220035 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:52:46.220035 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:52:46.220035 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:52:46.220035 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:52:46.220035 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:52:46.220035 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:52:46.220035 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:52:46.220035 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 55/106] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:52:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 491 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:52:46.224249 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece4270-b3ad-4aed-8fc7-2c11e4ad8602 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:46.224794 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece4270-b3ad-4aed-8fc7-2c11e4ad8602 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:46.225249 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece4270-b3ad-4aed-8fc7-2c11e4ad8602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:46.260664 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-2ece4270-b3ad-4aed-8fc7-2c11e4ad8602 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:46.262484 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-2ece4270-b3ad-4aed-8fc7-2c11e4ad8602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:46.364803 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 56/107] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:52:46.369958 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece4270-b3ad-4aed-8fc7-2c11e4ad8602 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:46.480339 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6b2915-0ede-4645-9734-94926fe46223 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:46.524109 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b2050-b3c5-4388-aa75-526be44533c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:46.524989 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b2050-b3c5-4388-aa75-526be44533c1 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:46.525296 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b2050-b3c5-4388-aa75-526be44533c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:46.525671 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b2050-b3c5-4388-aa75-526be44533c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:46.608771 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:46.620366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b2050-b3c5-4388-aa75-526be44533c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:46.621884 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b2050-b3c5-4388-aa75-526be44533c1 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:46.673801 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:46.696963 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece4270-b3ad-4aed-8fc7-2c11e4ad8602 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:46.733978 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22692cc-93e8-407a-b7fb-3d8ec0c2d4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:46.734833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22692cc-93e8-407a-b7fb-3d8ec0c2d4c1 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:46.735152 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22692cc-93e8-407a-b7fb-3d8ec0c2d4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:46.735428 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22692cc-93e8-407a-b7fb-3d8ec0c2d4c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:46.736165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22692cc-93e8-407a-b7fb-3d8ec0c2d4c1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:52:46.736516 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22692cc-93e8-407a-b7fb-3d8ec0c2d4c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:46.737949 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22692cc-93e8-407a-b7fb-3d8ec0c2d4c1 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:46.747181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 57/108] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 13:52:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:46.751955 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22692cc-93e8-407a-b7fb-3d8ec0c2d4c1 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:46.759637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 52/109] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:52:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 616 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:52:46.761405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55341659-f9b7-44db-b926-907506f87012 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:46.761895 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55341659-f9b7-44db-b926-907506f87012 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:46.762380 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55341659-f9b7-44db-b926-907506f87012 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:46.763863 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 53/110] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:52:46.764903 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55341659-f9b7-44db-b926-907506f87012 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:46.857270 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:46.867758 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22692cc-93e8-407a-b7fb-3d8ec0c2d4c1 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:46.886812 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55341659-f9b7-44db-b926-907506f87012 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:46.905322 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5df56d-3cbd-4b23-9914-f226fb77fe0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:46.906001 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5df56d-3cbd-4b23-9914-f226fb77fe0e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:46.906391 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5df56d-3cbd-4b23-9914-f226fb77fe0e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:46.906668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5df56d-3cbd-4b23-9914-f226fb77fe0e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:46.907896 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5df56d-3cbd-4b23-9914-f226fb77fe0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:46.909499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5df56d-3cbd-4b23-9914-f226fb77fe0e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:46.911142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 58/111] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:52:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:46.917078 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f03bd7-ee50-4f00-8ed1-4e889a460dee None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:46.917371 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f03bd7-ee50-4f00-8ed1-4e889a460dee None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:46.917681 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f03bd7-ee50-4f00-8ed1-4e889a460dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:46.929692 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d90d1c-a01a-4cb0-983d-da180ded22c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:46.930865 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d90d1c-a01a-4cb0-983d-da180ded22c2 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:46.931352 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d90d1c-a01a-4cb0-983d-da180ded22c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:46.931817 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d90d1c-a01a-4cb0-983d-da180ded22c2 None admin] PATH_INFO: `/v3/users/0cd5be8f747c4089988de317f6307400` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:46.933574 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d90d1c-a01a-4cb0-983d-da180ded22c2 None admin] RBAC: Authorizing `identity:get_user(user_id=0cd5be8f747c4089988de317f6307400)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:46.935486 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d90d1c-a01a-4cb0-983d-da180ded22c2 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:46.937809 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 54/112] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:52:46 2022] GET /identity/v3/users/0cd5be8f747c4089988de317f6307400 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:46.942990 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-26f03bd7-ee50-4f00-8ed1-4e889a460dee None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:46.942990 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-26f03bd7-ee50-4f00-8ed1-4e889a460dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:46.943791 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d90d1c-a01a-4cb0-983d-da180ded22c2 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:46.951001 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:47.015634 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 59/113] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:52:47.016731 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f03bd7-ee50-4f00-8ed1-4e889a460dee None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:47.066937 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d90d1c-a01a-4cb0-983d-da180ded22c2 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:47.110172 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae3202-0ff3-4ff6-afd0-b2169393ec61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:47.111299 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae3202-0ff3-4ff6-afd0-b2169393ec61 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:47.111858 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae3202-0ff3-4ff6-afd0-b2169393ec61 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:47.112370 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae3202-0ff3-4ff6-afd0-b2169393ec61 None admin] PATH_INFO: `/v3/projects/c18388255f054822b773ec88708292da` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:47.114455 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae3202-0ff3-4ff6-afd0-b2169393ec61 None admin] RBAC: Authorizing `identity:get_project(project_id=c18388255f054822b773ec88708292da)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:47.116242 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae3202-0ff3-4ff6-afd0-b2169393ec61 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:47.118545 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 55/114] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:52:46 2022] GET /identity/v3/projects/c18388255f054822b773ec88708292da => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:47.119567 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae3202-0ff3-4ff6-afd0-b2169393ec61 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:47.142895 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:47.145967 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f03bd7-ee50-4f00-8ed1-4e889a460dee None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:47.190134 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48933ad-536c-4966-b09b-f39034ad73ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:47.190857 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48933ad-536c-4966-b09b-f39034ad73ad None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:47.191100 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48933ad-536c-4966-b09b-f39034ad73ad None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:47.191421 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48933ad-536c-4966-b09b-f39034ad73ad None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:47.192241 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48933ad-536c-4966-b09b-f39034ad73ad None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:47.194391 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48933ad-536c-4966-b09b-f39034ad73ad None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:47.213160 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:47.230586 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 60/115] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Aug 1 13:52:47 2022] POST /identity/v3/projects => generated 302 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:52:47.238324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48933ad-536c-4966-b09b-f39034ad73ad None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:47.248070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae3202-0ff3-4ff6-afd0-b2169393ec61 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:47.326240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ccb94-bfa7-46d3-b177-d8e44c5d9bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:47.327133 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ccb94-bfa7-46d3-b177-d8e44c5d9bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:47.327419 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ccb94-bfa7-46d3-b177-d8e44c5d9bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:47.327733 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ccb94-bfa7-46d3-b177-d8e44c5d9bf2 None admin] PATH_INFO: `/v3/projects/c18388255f054822b773ec88708292da` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:47.329851 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ccb94-bfa7-46d3-b177-d8e44c5d9bf2 None admin] RBAC: Authorizing `identity:get_project(project_id=c18388255f054822b773ec88708292da)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:47.332679 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ccb94-bfa7-46d3-b177-d8e44c5d9bf2 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:47.340315 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 56/116] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:52:47 2022] GET /identity/v3/projects/c18388255f054822b773ec88708292da => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:47.340315 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ccb94-bfa7-46d3-b177-d8e44c5d9bf2 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:47.405227 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48933ad-536c-4966-b09b-f39034ad73ad None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:47.466113 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e350756-8418-4ad5-844b-490afbd96679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:47.466841 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e350756-8418-4ad5-844b-490afbd96679 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:47.467123 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e350756-8418-4ad5-844b-490afbd96679 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:47.467407 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e350756-8418-4ad5-844b-490afbd96679 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:47.477003 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:47.482180 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e350756-8418-4ad5-844b-490afbd96679 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:52:47.487953 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-0e350756-8418-4ad5-844b-490afbd96679 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:52:47.487953 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:52:47.487953 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:52:47.487953 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:52:47.487953 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:52:47.487953 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:52:47.487953 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:52:47.487953 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:52:47.487953 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:52:47.487953 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:52:47.487953 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:52:47.487953 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:52:47.487953 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:52:47.487953 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:52:47.487953 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:52:47.487953 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:52:47.489536 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:52:47.489536 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:52:47.489536 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:52:47.489536 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:52:47.489536 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:52:47.489536 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:52:47.489536 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:52:47.489536 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:52:47.489536 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:52:47.489536 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:52:47.489536 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:52:47.489536 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:52:47.489536 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:52:47.489536 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:52:47.489536 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:52:47.489536 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:52:47.490499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:52:47.490499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:52:47.490499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:52:47.490499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:52:47.490499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:52:47.490499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:52:47.490499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:52:47.490499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:52:47.490499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:52:47.490499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:52:47.490499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:52:47.490499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:52:47.490499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 61/117] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 13:52:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:52:47.491386 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f58ce-f0a5-497e-9582-32d8779683d2 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:47.491386 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f58ce-f0a5-497e-9582-32d8779683d2 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:47.491386 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f58ce-f0a5-497e-9582-32d8779683d2 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:47.494093 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 62/118] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:52:47.498254 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f58ce-f0a5-497e-9582-32d8779683d2 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:47.521774 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ccb94-bfa7-46d3-b177-d8e44c5d9bf2 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:47.579274 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd4b1d-18e9-410c-bc4a-46e773a87caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:47.579998 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd4b1d-18e9-410c-bc4a-46e773a87caf None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:47.580308 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd4b1d-18e9-410c-bc4a-46e773a87caf None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:47.580590 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd4b1d-18e9-410c-bc4a-46e773a87caf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:47.581405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd4b1d-18e9-410c-bc4a-46e773a87caf None admin] RBAC: Adding query filter params (role.id=9db7d52dd9514261a5a52591fbf1feef, scope.project.id=c18388255f054822b773ec88708292da, user.id=70ba8bcd60de4443b95ce1e8d22ff2b4) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:52:47.581720 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd4b1d-18e9-410c-bc4a-46e773a87caf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:47.583230 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd4b1d-18e9-410c-bc4a-46e773a87caf None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:47.589399 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 57/119] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:52:47 2022] GET /identity/v3/role_assignments?user.id=70ba8bcd60de4443b95ce1e8d22ff2b4&scope.project.id=c18388255f054822b773ec88708292da&role.id=9db7d52dd9514261a5a52591fbf1feef => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:47.590495 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becadaca-3b5c-4253-aa9a-3bc64d1c51b0 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:47.590786 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becadaca-3b5c-4253-aa9a-3bc64d1c51b0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:47.591063 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becadaca-3b5c-4253-aa9a-3bc64d1c51b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:47.595610 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:47.620581 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-becadaca-3b5c-4253-aa9a-3bc64d1c51b0 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:47.622895 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-becadaca-3b5c-4253-aa9a-3bc64d1c51b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:47.662800 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f58ce-f0a5-497e-9582-32d8779683d2 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:47.701752 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5cc1ce-ab84-460f-b973-309f760b2b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:47.702465 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cc1ce-ab84-460f-b973-309f760b2b2e None admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:47.702748 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cc1ce-ab84-460f-b973-309f760b2b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:47.703069 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cc1ce-ab84-460f-b973-309f760b2b2e None admin] PATH_INFO: `/v3/projects/c18388255f054822b773ec88708292da/users/0cd5be8f747c4089988de317f6307400/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:47.705221 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5cc1ce-ab84-460f-b973-309f760b2b2e None admin] RBAC: Authorizing `identity:create_grant(project_id=c18388255f054822b773ec88708292da, user_id=0cd5be8f747c4089988de317f6307400, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:47.706725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5cc1ce-ab84-460f-b973-309f760b2b2e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:47.725985 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 58/120] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:52:47.725985 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becadaca-3b5c-4253-aa9a-3bc64d1c51b0 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:47.730883 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 63/121] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 13:52:47 2022] PUT /identity/v3/projects/c18388255f054822b773ec88708292da/users/0cd5be8f747c4089988de317f6307400/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:52:47.732231 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58255b-a359-45ba-83ea-fdb907452f6a None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:47.732547 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58255b-a359-45ba-83ea-fdb907452f6a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:47.732919 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58255b-a359-45ba-83ea-fdb907452f6a None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:47.734342 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 64/122] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:52:47.746163 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2335c1-a324-45c1-b0a7-450dbaf7f2f2 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:47.746635 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2335c1-a324-45c1-b0a7-450dbaf7f2f2 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:47.746734 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2335c1-a324-45c1-b0a7-450dbaf7f2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:47.782457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-7f2335c1-a324-45c1-b0a7-450dbaf7f2f2 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:47.783903 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7f2335c1-a324-45c1-b0a7-450dbaf7f2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:47.881066 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:47.937358 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becadaca-3b5c-4253-aa9a-3bc64d1c51b0 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:47.957834 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 65/123] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:52:47.958510 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2335c1-a324-45c1-b0a7-450dbaf7f2f2 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:48.006396 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7532113d-5322-498e-ba7c-1b25b827106f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:48.007467 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7532113d-5322-498e-ba7c-1b25b827106f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:48.007964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7532113d-5322-498e-ba7c-1b25b827106f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:48.008495 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7532113d-5322-498e-ba7c-1b25b827106f None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:48.009842 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7532113d-5322-498e-ba7c-1b25b827106f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:52:48.010391 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7532113d-5322-498e-ba7c-1b25b827106f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:48.013119 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7532113d-5322-498e-ba7c-1b25b827106f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:48.036385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 59/124] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:52:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:48.037172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:48.037538 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7532113d-5322-498e-ba7c-1b25b827106f None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:48.172308 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2335c1-a324-45c1-b0a7-450dbaf7f2f2 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:48.252102 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7532113d-5322-498e-ba7c-1b25b827106f None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:48.271495 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376cbe8-3405-433e-ba9d-670350c3a8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:48.272607 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376cbe8-3405-433e-ba9d-670350c3a8ca None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:48.273096 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376cbe8-3405-433e-ba9d-670350c3a8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:48.273569 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376cbe8-3405-433e-ba9d-670350c3a8ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:48.304623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff47fa6-7702-4a69-9629-58e635d223b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:48.306042 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff47fa6-7702-4a69-9629-58e635d223b0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:48.306594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff47fa6-7702-4a69-9629-58e635d223b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:48.307170 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff47fa6-7702-4a69-9629-58e635d223b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:48.309715 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c376cbe8-3405-433e-ba9d-670350c3a8ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:52:48.316978 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-c376cbe8-3405-433e-ba9d-670350c3a8ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:52:48.316978 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:52:48.316978 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:52:48.316978 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:52:48.316978 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:52:48.316978 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:52:48.316978 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:52:48.316978 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:52:48.316978 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:52:48.316978 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:52:48.316978 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:52:48.316978 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:52:48.316978 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:52:48.316978 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:52:48.316978 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:52:48.316978 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:52:48.317931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:52:48.317931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:52:48.317931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:52:48.317931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:52:48.317931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:52:48.317931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:52:48.317931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:52:48.317931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:52:48.317931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:52:48.317931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:52:48.317931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:52:48.317931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:52:48.317931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:52:48.317931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:52:48.317931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:52:48.317931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:52:48.318907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:52:48.318907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:52:48.318907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:52:48.318907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:52:48.318907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:52:48.318907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:52:48.318907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:52:48.318907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:52:48.318907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:52:48.318907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:52:48.318907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:52:48.318907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:52:48.323799 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 66/125] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:52:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:52:48.324463 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:48.325396 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376cbe8-3405-433e-ba9d-670350c3a8ca None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:48.353623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff47fa6-7702-4a69-9629-58e635d223b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:52:48.364588 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-0ff47fa6-7702-4a69-9629-58e635d223b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:52:48.364588 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:52:48.364588 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:52:48.364588 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:52:48.364588 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:52:48.364588 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:52:48.364588 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:52:48.364588 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:52:48.364588 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:52:48.364588 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:52:48.364588 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:52:48.364588 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:52:48.364588 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:52:48.364588 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:52:48.364588 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:52:48.364588 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:52:48.366159 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:52:48.366159 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:52:48.366159 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:52:48.366159 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:52:48.366159 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:52:48.366159 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:52:48.366159 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:52:48.366159 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:52:48.366159 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:52:48.366159 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:52:48.366159 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:52:48.366159 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:52:48.366159 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:52:48.366159 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:52:48.366159 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:52:48.366159 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:52:48.367519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:52:48.367519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:52:48.367519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:52:48.367519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:52:48.367519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:52:48.367519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:52:48.367519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:52:48.367519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:52:48.367519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:52:48.367519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:52:48.367519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:52:48.367519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:52:48.367519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 60/126] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:52:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:52:48.367519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff47fa6-7702-4a69-9629-58e635d223b0 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:48.588904 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff47fa6-7702-4a69-9629-58e635d223b0 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:48.640533 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63734589-216a-47ad-b8ae-fe6a09fe1a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:48.641452 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63734589-216a-47ad-b8ae-fe6a09fe1a45 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:48.641784 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63734589-216a-47ad-b8ae-fe6a09fe1a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:48.642130 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63734589-216a-47ad-b8ae-fe6a09fe1a45 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:48.643405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63734589-216a-47ad-b8ae-fe6a09fe1a45 None admin] RBAC: Adding query filter params (name=member) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:52:48.643836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63734589-216a-47ad-b8ae-fe6a09fe1a45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:48.645425 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63734589-216a-47ad-b8ae-fe6a09fe1a45 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:48.650994 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376cbe8-3405-433e-ba9d-670350c3a8ca None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:48.655562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 61/127] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 13:52:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:48.656832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63734589-216a-47ad-b8ae-fe6a09fe1a45 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:48.669052 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:48.740154 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83395a07-0f03-4854-a65c-70470556c210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:48.740958 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83395a07-0f03-4854-a65c-70470556c210 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:48.741265 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83395a07-0f03-4854-a65c-70470556c210 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:48.741550 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83395a07-0f03-4854-a65c-70470556c210 None admin] PATH_INFO: `/v3/users/0cd5be8f747c4089988de317f6307400` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:48.743173 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83395a07-0f03-4854-a65c-70470556c210 None admin] RBAC: Authorizing `identity:get_user(user_id=0cd5be8f747c4089988de317f6307400)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:48.744464 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83395a07-0f03-4854-a65c-70470556c210 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:48.746169 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 67/128] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:52:48 2022] GET /identity/v3/users/0cd5be8f747c4089988de317f6307400 => generated 275 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:48.748161 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83395a07-0f03-4854-a65c-70470556c210 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:48.784916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:48.838386 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63734589-216a-47ad-b8ae-fe6a09fe1a45 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:48.882844 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ac6d6-393d-4290-a3c6-4dac7ec41bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:48.883674 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83395a07-0f03-4854-a65c-70470556c210 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:48.884053 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ac6d6-393d-4290-a3c6-4dac7ec41bca None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:48.884534 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ac6d6-393d-4290-a3c6-4dac7ec41bca None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:48.884983 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ac6d6-393d-4290-a3c6-4dac7ec41bca None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:48.886009 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ac6d6-393d-4290-a3c6-4dac7ec41bca None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:52:48.886558 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ac6d6-393d-4290-a3c6-4dac7ec41bca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:48.888303 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ac6d6-393d-4290-a3c6-4dac7ec41bca None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:48.899510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 62/129] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:52:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:48.900826 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ac6d6-393d-4290-a3c6-4dac7ec41bca None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:48.956427 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393563e4-332a-408b-a6c6-787060e8b863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:48.957130 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393563e4-332a-408b-a6c6-787060e8b863 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:48.957421 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393563e4-332a-408b-a6c6-787060e8b863 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:48.957725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393563e4-332a-408b-a6c6-787060e8b863 None admin] PATH_INFO: `/v3/users/0cd5be8f747c4089988de317f6307400` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:48.959664 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393563e4-332a-408b-a6c6-787060e8b863 None admin] RBAC: Authorizing `identity:get_user(user_id=0cd5be8f747c4089988de317f6307400)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:48.961526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393563e4-332a-408b-a6c6-787060e8b863 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:48.963505 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 68/130] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:52:48 2022] GET /identity/v3/users/0cd5be8f747c4089988de317f6307400 => generated 275 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:48.967484 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393563e4-332a-408b-a6c6-787060e8b863 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:49.033614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:49.104953 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393563e4-332a-408b-a6c6-787060e8b863 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:49.148127 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdab26d-6adc-452b-a821-f1754bef236e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:49.148850 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdab26d-6adc-452b-a821-f1754bef236e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:49.149138 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdab26d-6adc-452b-a821-f1754bef236e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:49.149419 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdab26d-6adc-452b-a821-f1754bef236e None admin] PATH_INFO: `/v3/users/bf301c0e8be34b7fa129e68f7514a379` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:49.150843 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdab26d-6adc-452b-a821-f1754bef236e None admin] RBAC: Authorizing `identity:get_user(user_id=bf301c0e8be34b7fa129e68f7514a379)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:49.152378 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdab26d-6adc-452b-a821-f1754bef236e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:49.154849 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 69/131] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:52:48 2022] GET /identity/v3/users/bf301c0e8be34b7fa129e68f7514a379 => generated 247 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:49.155629 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdab26d-6adc-452b-a821-f1754bef236e None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:49.242375 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ac6d6-393d-4290-a3c6-4dac7ec41bca None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:49.282681 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdab26d-6adc-452b-a821-f1754bef236e None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:49.302285 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:49.313050 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:49.336132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006f3405-051a-46bd-8988-d0c3412e5f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:49.337242 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f3405-051a-46bd-8988-d0c3412e5f00 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:49.337562 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f3405-051a-46bd-8988-d0c3412e5f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:49.338029 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f3405-051a-46bd-8988-d0c3412e5f00 None admin] PATH_INFO: `/v3/projects/fd7418cfcedb4164bed000f138751888` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:49.339789 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006f3405-051a-46bd-8988-d0c3412e5f00 None admin] RBAC: Authorizing `identity:get_project(project_id=fd7418cfcedb4164bed000f138751888)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:49.342202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006f3405-051a-46bd-8988-d0c3412e5f00 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:49.345016 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 70/132] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:52:49 2022] GET /identity/v3/projects/fd7418cfcedb4164bed000f138751888 => generated 312 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:49.346016 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006f3405-051a-46bd-8988-d0c3412e5f00 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:49.384357 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934354be-21f6-4c4a-9394-ec5f535a7eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:49.384970 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934354be-21f6-4c4a-9394-ec5f535a7eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:49.385274 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934354be-21f6-4c4a-9394-ec5f535a7eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:49.385689 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934354be-21f6-4c4a-9394-ec5f535a7eb8 None admin] PATH_INFO: `/v3/projects/c18388255f054822b773ec88708292da` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:49.389183 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934354be-21f6-4c4a-9394-ec5f535a7eb8 None admin] RBAC: Authorizing `identity:get_project(project_id=c18388255f054822b773ec88708292da)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:49.391185 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934354be-21f6-4c4a-9394-ec5f535a7eb8 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:49.394018 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 63/133] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:52:48 2022] GET /identity/v3/projects/c18388255f054822b773ec88708292da => generated 298 bytes in 493 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:49.405812 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934354be-21f6-4c4a-9394-ec5f535a7eb8 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:49.555783 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006f3405-051a-46bd-8988-d0c3412e5f00 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:49.613119 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934354be-21f6-4c4a-9394-ec5f535a7eb8 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:49.623716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a56141-baec-46c9-8703-49f8a98aee86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:49.624481 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a56141-baec-46c9-8703-49f8a98aee86 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:49.624922 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a56141-baec-46c9-8703-49f8a98aee86 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:49.625320 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a56141-baec-46c9-8703-49f8a98aee86 None admin] PATH_INFO: `/v3/projects/c18388255f054822b773ec88708292da` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:49.626865 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a56141-baec-46c9-8703-49f8a98aee86 None admin] RBAC: Authorizing `identity:get_project(project_id=c18388255f054822b773ec88708292da)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:49.628569 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a56141-baec-46c9-8703-49f8a98aee86 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:49.630562 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 71/134] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:52:49 2022] GET /identity/v3/projects/c18388255f054822b773ec88708292da => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:49.631060 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a56141-baec-46c9-8703-49f8a98aee86 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:49.695825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3755c5-75c8-4601-8911-dc9fd6fde164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:49.696504 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3755c5-75c8-4601-8911-dc9fd6fde164 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:49.696788 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3755c5-75c8-4601-8911-dc9fd6fde164 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:49.697081 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3755c5-75c8-4601-8911-dc9fd6fde164 None admin] PATH_INFO: `/v3/projects/fd7418cfcedb4164bed000f138751888/users/0cd5be8f747c4089988de317f6307400/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:49.701400 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3755c5-75c8-4601-8911-dc9fd6fde164 None admin] RBAC: Authorizing `identity:create_grant(project_id=fd7418cfcedb4164bed000f138751888, user_id=0cd5be8f747c4089988de317f6307400, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:49.702906 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3755c5-75c8-4601-8911-dc9fd6fde164 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:49.730602 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 64/135] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 13:52:49 2022] PUT /identity/v3/projects/fd7418cfcedb4164bed000f138751888/users/0cd5be8f747c4089988de317f6307400/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:52:49.731290 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3755c5-75c8-4601-8911-dc9fd6fde164 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:49.744712 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:49.866318 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:49.882570 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a56141-baec-46c9-8703-49f8a98aee86 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:49.926610 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3755c5-75c8-4601-8911-dc9fd6fde164 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:49.943579 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0a672-01b1-4dc1-b30e-858a68b452c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:49.944705 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0a672-01b1-4dc1-b30e-858a68b452c4 None admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:49.944989 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0a672-01b1-4dc1-b30e-858a68b452c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:49.945262 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0a672-01b1-4dc1-b30e-858a68b452c4 None admin] PATH_INFO: `/v3/projects/c18388255f054822b773ec88708292da/users/0cd5be8f747c4089988de317f6307400/roles/60efb70fa7354ebfb04aa5e9bdad2a78` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:49.947848 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0a672-01b1-4dc1-b30e-858a68b452c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=c18388255f054822b773ec88708292da, user_id=0cd5be8f747c4089988de317f6307400, role_id=60efb70fa7354ebfb04aa5e9bdad2a78)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:49.949470 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0a672-01b1-4dc1-b30e-858a68b452c4 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:49.972472 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787a66f-2eed-4e8a-8d31-62172b31ea4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:49.974196 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787a66f-2eed-4e8a-8d31-62172b31ea4a None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:49.975048 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787a66f-2eed-4e8a-8d31-62172b31ea4a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:49.975908 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787a66f-2eed-4e8a-8d31-62172b31ea4a None admin] PATH_INFO: `/v3/projects/c18388255f054822b773ec88708292da/users/bf301c0e8be34b7fa129e68f7514a379/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:49.981716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 72/136] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 13:52:49 2022] PUT /identity/v3/projects/c18388255f054822b773ec88708292da/users/0cd5be8f747c4089988de317f6307400/roles/60efb70fa7354ebfb04aa5e9bdad2a78 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:52:49.982751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787a66f-2eed-4e8a-8d31-62172b31ea4a None admin] RBAC: Authorizing `identity:create_grant(project_id=c18388255f054822b773ec88708292da, user_id=bf301c0e8be34b7fa129e68f7514a379, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:49.986361 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787a66f-2eed-4e8a-8d31-62172b31ea4a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:50.009438 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 65/137] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 13:52:49 2022] PUT /identity/v3/projects/c18388255f054822b773ec88708292da/users/bf301c0e8be34b7fa129e68f7514a379/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:52:50.157696 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa83119-3d2d-4019-bd07-eef2b013d7a8 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:50.157990 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa83119-3d2d-4019-bd07-eef2b013d7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:50.158279 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa83119-3d2d-4019-bd07-eef2b013d7a8 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:50.159771 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 73/138] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:52:50.165793 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3144a1-2728-40a1-9685-89df7ce648e9 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:50.166251 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3144a1-2728-40a1-9685-89df7ce648e9 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:50.166635 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3144a1-2728-40a1-9685-89df7ce648e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:50.189456 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-1f3144a1-2728-40a1-9685-89df7ce648e9 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:50.190942 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1f3144a1-2728-40a1-9685-89df7ce648e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:50.284597 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 66/139] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:52:50.293561 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c45d99-3cea-4725-b742-92562d4ae661 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:50.294154 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c45d99-3cea-4725-b742-92562d4ae661 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:50.294337 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c45d99-3cea-4725-b742-92562d4ae661 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:50.296196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 74/140] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:52:50.303687 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1210c8a3-f2bc-4356-a95d-2fe3c4e3fd78 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:50.303993 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1210c8a3-f2bc-4356-a95d-2fe3c4e3fd78 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:50.304764 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1210c8a3-f2bc-4356-a95d-2fe3c4e3fd78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:50.315260 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c45d99-3cea-4725-b742-92562d4ae661 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:50.356317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-1210c8a3-f2bc-4356-a95d-2fe3c4e3fd78 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:50.360183 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1210c8a3-f2bc-4356-a95d-2fe3c4e3fd78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:50.428662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:50.540892 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 67/141] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:52:50.561167 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c45d99-3cea-4725-b742-92562d4ae661 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:50.576217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:50.585450 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1210c8a3-f2bc-4356-a95d-2fe3c4e3fd78 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:50.626136 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd1eda2-5a3b-4c4b-8a07-c812a8faff05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:50.627230 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1eda2-5a3b-4c4b-8a07-c812a8faff05 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:50.627686 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1eda2-5a3b-4c4b-8a07-c812a8faff05 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:50.628131 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1eda2-5a3b-4c4b-8a07-c812a8faff05 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:50.647295 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd1eda2-5a3b-4c4b-8a07-c812a8faff05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:52:50.656583 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-5cd1eda2-5a3b-4c4b-8a07-c812a8faff05 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:52:50.656583 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:52:50.656583 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:52:50.656583 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:52:50.656583 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:52:50.656583 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:52:50.656583 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:52:50.656583 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:52:50.656583 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:52:50.656583 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:52:50.656583 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:52:50.656583 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:52:50.656583 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:52:50.656583 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:52:50.656583 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:52:50.656583 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:52:50.657509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:52:50.657509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:52:50.657509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:52:50.657509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:52:50.657509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:52:50.657509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:52:50.657509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:52:50.657509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:52:50.657509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:52:50.657509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:52:50.657509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:52:50.657509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:52:50.657509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:52:50.657509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:52:50.657509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:52:50.657509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:52:50.658259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:52:50.658259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:52:50.658259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:52:50.658259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:52:50.658259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:52:50.658259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:52:50.658259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:52:50.658259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:52:50.658259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:52:50.658259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:52:50.658259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:52:50.658259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:52:50.658832 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 75/142] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:52:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:52:50.667693 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd1eda2-5a3b-4c4b-8a07-c812a8faff05 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:50.826568 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1210c8a3-f2bc-4356-a95d-2fe3c4e3fd78 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:50.871202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd1eda2-5a3b-4c4b-8a07-c812a8faff05 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:50.915878 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346ab94-0396-49fd-8425-a8035ddf4dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:50.916921 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346ab94-0396-49fd-8425-a8035ddf4dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:50.917204 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346ab94-0396-49fd-8425-a8035ddf4dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:50.917540 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346ab94-0396-49fd-8425-a8035ddf4dc0 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:50.918449 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5346ab94-0396-49fd-8425-a8035ddf4dc0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:52:50.918806 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5346ab94-0396-49fd-8425-a8035ddf4dc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:50.920545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5346ab94-0396-49fd-8425-a8035ddf4dc0 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:50.937025 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 76/143] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 13:52:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:50.943050 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346ab94-0396-49fd-8425-a8035ddf4dc0 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:50.953201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9808eb61-56f7-4fea-8bd1-f219fe9d718b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:50.954266 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808eb61-56f7-4fea-8bd1-f219fe9d718b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:50.954738 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808eb61-56f7-4fea-8bd1-f219fe9d718b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:50.955222 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808eb61-56f7-4fea-8bd1-f219fe9d718b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:50.960579 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9808eb61-56f7-4fea-8bd1-f219fe9d718b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:50.962422 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9808eb61-56f7-4fea-8bd1-f219fe9d718b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:50.964778 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 68/144] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:52:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:50.976298 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82944f8-a573-4667-bf97-02387cc6fe70 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:50.977225 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82944f8-a573-4667-bf97-02387cc6fe70 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:50.977701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82944f8-a573-4667-bf97-02387cc6fe70 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:50.979393 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 69/145] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:50 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:52:50.980511 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82944f8-a573-4667-bf97-02387cc6fe70 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:51.009381 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:51.080298 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:51.169452 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82944f8-a573-4667-bf97-02387cc6fe70 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:51.242982 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14626b2-9e71-429a-a14c-b5b3db9a1899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:51.244030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14626b2-9e71-429a-a14c-b5b3db9a1899 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:51.244547 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14626b2-9e71-429a-a14c-b5b3db9a1899 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:51.245034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14626b2-9e71-429a-a14c-b5b3db9a1899 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:51.246221 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14626b2-9e71-429a-a14c-b5b3db9a1899 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:51.248528 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14626b2-9e71-429a-a14c-b5b3db9a1899 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:51.254426 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346ab94-0396-49fd-8425-a8035ddf4dc0 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:51.292967 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 70/146] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 13:52:50 2022] POST /identity/v3/users => generated 283 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:52:51.300383 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08bff80-6bef-4831-9520-93ec57ed05e1 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:51.300383 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08bff80-6bef-4831-9520-93ec57ed05e1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:51.300383 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08bff80-6bef-4831-9520-93ec57ed05e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:51.353811 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c08bff80-6bef-4831-9520-93ec57ed05e1 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:51.355010 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c08bff80-6bef-4831-9520-93ec57ed05e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:51.401066 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f1586-0488-4981-8184-4ad37012b397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:51.401785 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f1586-0488-4981-8184-4ad37012b397 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:51.402177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f1586-0488-4981-8184-4ad37012b397 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:51.403226 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f1586-0488-4981-8184-4ad37012b397 None admin] PATH_INFO: `/v3/users/70ba8bcd60de4443b95ce1e8d22ff2b4` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:51.409375 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f1586-0488-4981-8184-4ad37012b397 None admin] RBAC: Authorizing `identity:get_user(user_id=70ba8bcd60de4443b95ce1e8d22ff2b4)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:51.411710 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f1586-0488-4981-8184-4ad37012b397 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:51.413937 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 77/147] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:52:50 2022] GET /identity/v3/users/70ba8bcd60de4443b95ce1e8d22ff2b4 => generated 289 bytes in 471 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:52:51.422703 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f1586-0488-4981-8184-4ad37012b397 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:51.530621 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 71/148] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:52:51.602893 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08bff80-6bef-4831-9520-93ec57ed05e1 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:51.639307 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:51.676304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:51.690340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f1586-0488-4981-8184-4ad37012b397 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:51.770345 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1183a-623e-4139-83f9-7d57efd8774c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:51.771039 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1183a-623e-4139-83f9-7d57efd8774c None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:51.771324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1183a-623e-4139-83f9-7d57efd8774c None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:51.771605 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1183a-623e-4139-83f9-7d57efd8774c None admin] PATH_INFO: `/v3/projects/c18388255f054822b773ec88708292da` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:51.773279 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1183a-623e-4139-83f9-7d57efd8774c None admin] RBAC: Authorizing `identity:get_project(project_id=c18388255f054822b773ec88708292da)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:51.774820 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1183a-623e-4139-83f9-7d57efd8774c None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:51.776747 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 78/149] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:52:51 2022] GET /identity/v3/projects/c18388255f054822b773ec88708292da => generated 298 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:52:51.782891 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1183a-623e-4139-83f9-7d57efd8774c None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:51.795517 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08bff80-6bef-4831-9520-93ec57ed05e1 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:51.869505 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6da31b-4d02-4aa1-a336-a4910be7a20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:51.870513 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6da31b-4d02-4aa1-a336-a4910be7a20f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:51.871035 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6da31b-4d02-4aa1-a336-a4910be7a20f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:51.871706 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6da31b-4d02-4aa1-a336-a4910be7a20f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:51.902297 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6da31b-4d02-4aa1-a336-a4910be7a20f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:52:51.907732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-be6da31b-4d02-4aa1-a336-a4910be7a20f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:52:51.907732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:52:51.907732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:52:51.907732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:52:51.907732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:52:51.907732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:52:51.907732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:52:51.907732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:52:51.907732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:52:51.907732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:52:51.907732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:52:51.907732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:52:51.907732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:52:51.907732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:52:51.907732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:52:51.907732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:52:51.908686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:52:51.908686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:52:51.908686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:52:51.908686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:52:51.908686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:52:51.908686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:52:51.908686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:52:51.908686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:52:51.908686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:52:51.908686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:52:51.908686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:52:51.908686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:52:51.908686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:52:51.908686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:52:51.908686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:52:51.908686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:52:51.909457 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:52:51.909457 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:52:51.909457 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:52:51.909457 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:52:51.909457 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:52:51.909457 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:52:51.909457 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:52:51.909457 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:52:51.909457 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:52:51.909457 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:52:51.909457 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:52:51.909457 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:52:51.910037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1183a-623e-4139-83f9-7d57efd8774c None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:51.920304 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 72/150] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:52:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:52:51.921821 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6da31b-4d02-4aa1-a336-a4910be7a20f None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:51.953981 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7879968a-c687-4298-977f-cc07a55ab877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:51.954756 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879968a-c687-4298-977f-cc07a55ab877 None admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:51.955043 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879968a-c687-4298-977f-cc07a55ab877 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:51.955321 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879968a-c687-4298-977f-cc07a55ab877 None admin] PATH_INFO: `/v3/projects/c18388255f054822b773ec88708292da/users/70ba8bcd60de4443b95ce1e8d22ff2b4/roles/9db7d52dd9514261a5a52591fbf1feef` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:51.960441 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7879968a-c687-4298-977f-cc07a55ab877 None admin] RBAC: Authorizing `identity:create_grant(project_id=c18388255f054822b773ec88708292da, user_id=70ba8bcd60de4443b95ce1e8d22ff2b4, role_id=9db7d52dd9514261a5a52591fbf1feef)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:51.962056 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7879968a-c687-4298-977f-cc07a55ab877 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:51.991247 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 79/151] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 13:52:51 2022] PUT /identity/v3/projects/c18388255f054822b773ec88708292da/users/70ba8bcd60de4443b95ce1e8d22ff2b4/roles/9db7d52dd9514261a5a52591fbf1feef => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:52:52.120719 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:52.149244 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6da31b-4d02-4aa1-a336-a4910be7a20f None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:52.196062 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c5676-dd69-4e9f-b82b-9f792379201e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:52.196658 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c5676-dd69-4e9f-b82b-9f792379201e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:52.196941 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c5676-dd69-4e9f-b82b-9f792379201e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:52.197219 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c5676-dd69-4e9f-b82b-9f792379201e None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:52.197975 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1c5676-dd69-4e9f-b82b-9f792379201e None admin] RBAC: Adding query filter params (name=member) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:52:52.198284 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1c5676-dd69-4e9f-b82b-9f792379201e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:52.199800 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1c5676-dd69-4e9f-b82b-9f792379201e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:52.209956 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 73/152] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 13:52:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:52.218215 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7879968a-c687-4298-977f-cc07a55ab877 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:52.277426 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:52.419071 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7879968a-c687-4298-977f-cc07a55ab877 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:52.459845 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7c126-8f4a-4504-b0f2-dd712a167d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:52.460515 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7c126-8f4a-4504-b0f2-dd712a167d09 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:52.460802 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7c126-8f4a-4504-b0f2-dd712a167d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:52.461079 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7c126-8f4a-4504-b0f2-dd712a167d09 None admin] PATH_INFO: `/v3/users/0cd5be8f747c4089988de317f6307400` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:52.462326 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7c126-8f4a-4504-b0f2-dd712a167d09 None admin] RBAC: Authorizing `identity:get_user(user_id=0cd5be8f747c4089988de317f6307400)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:52.463867 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7c126-8f4a-4504-b0f2-dd712a167d09 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:52.465633 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 80/153] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:52:52 2022] GET /identity/v3/users/0cd5be8f747c4089988de317f6307400 => generated 275 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:52.471576 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c5676-dd69-4e9f-b82b-9f792379201e None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:52.624149 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c5676-dd69-4e9f-b82b-9f792379201e None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:52.667417 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad816572-bd3d-4cfa-820a-5980c1c6641d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:52.668063 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad816572-bd3d-4cfa-820a-5980c1c6641d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:52.668375 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad816572-bd3d-4cfa-820a-5980c1c6641d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:52.668658 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad816572-bd3d-4cfa-820a-5980c1c6641d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:52.685278 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:52.752066 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad816572-bd3d-4cfa-820a-5980c1c6641d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:52.753615 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad816572-bd3d-4cfa-820a-5980c1c6641d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:52.872208 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 74/154] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:52:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:52:52.879037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7c126-8f4a-4504-b0f2-dd712a167d09 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:53.027416 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:53.034024 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7c126-8f4a-4504-b0f2-dd712a167d09 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:53.084177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fdf44-0a00-4dd3-a4e3-ce2cdf7eb208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:53.084924 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fdf44-0a00-4dd3-a4e3-ce2cdf7eb208 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:53.085220 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fdf44-0a00-4dd3-a4e3-ce2cdf7eb208 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:53.085491 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fdf44-0a00-4dd3-a4e3-ce2cdf7eb208 None admin] PATH_INFO: `/v3/projects/c18388255f054822b773ec88708292da` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:53.086766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fdf44-0a00-4dd3-a4e3-ce2cdf7eb208 None admin] RBAC: Authorizing `identity:get_project(project_id=c18388255f054822b773ec88708292da)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:53.088312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fdf44-0a00-4dd3-a4e3-ce2cdf7eb208 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:53.090373 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 81/155] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:52:52 2022] GET /identity/v3/projects/c18388255f054822b773ec88708292da => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:53.096915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad816572-bd3d-4cfa-820a-5980c1c6641d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:53.200653 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:53.256502 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad816572-bd3d-4cfa-820a-5980c1c6641d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:53.307223 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf9ac8-aa56-4518-8e21-a7320edb0856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:53.307772 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf9ac8-aa56-4518-8e21-a7320edb0856 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:53.308054 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf9ac8-aa56-4518-8e21-a7320edb0856 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:53.308332 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf9ac8-aa56-4518-8e21-a7320edb0856 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:53.309140 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bf9ac8-aa56-4518-8e21-a7320edb0856 None admin] RBAC: Adding query filter params (role.id=75aff8cb4cf748a2adbd6c3f141fe8ac, scope.project.id=c18388255f054822b773ec88708292da, user.id=0cd5be8f747c4089988de317f6307400) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:52:53.309452 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bf9ac8-aa56-4518-8e21-a7320edb0856 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:53.310925 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bf9ac8-aa56-4518-8e21-a7320edb0856 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:53.317981 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 75/156] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:52:53 2022] GET /identity/v3/role_assignments?user.id=0cd5be8f747c4089988de317f6307400&scope.project.id=c18388255f054822b773ec88708292da&role.id=75aff8cb4cf748a2adbd6c3f141fe8ac => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:53.388965 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce64f9b-08d1-4c6c-a984-234decf80244 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:53.389283 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce64f9b-08d1-4c6c-a984-234decf80244 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:53.389526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce64f9b-08d1-4c6c-a984-234decf80244 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:53.391213 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 82/157] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:52:53.397480 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e8d6a-f1cb-43d8-9ef3-73015ac384f0 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:53.397787 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e8d6a-f1cb-43d8-9ef3-73015ac384f0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:53.398044 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e8d6a-f1cb-43d8-9ef3-73015ac384f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:53.427417 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c28e8d6a-f1cb-43d8-9ef3-73015ac384f0 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:53.428938 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c28e8d6a-f1cb-43d8-9ef3-73015ac384f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:53.533454 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 76/158] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:52:53.584763 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce64f9b-08d1-4c6c-a984-234decf80244 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:53.736806 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce64f9b-08d1-4c6c-a984-234decf80244 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:53.788766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a4656-5bb3-40d2-8a7a-222f77de0e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:53.789508 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a4656-5bb3-40d2-8a7a-222f77de0e1c None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:53.789793 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a4656-5bb3-40d2-8a7a-222f77de0e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:53.790070 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a4656-5bb3-40d2-8a7a-222f77de0e1c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:53.809338 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15a4656-5bb3-40d2-8a7a-222f77de0e1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:52:53.817209 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-f15a4656-5bb3-40d2-8a7a-222f77de0e1c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:52:53.817209 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:52:53.817209 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:52:53.817209 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:52:53.817209 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:52:53.817209 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:52:53.817209 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:52:53.817209 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:52:53.817209 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:52:53.817209 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:52:53.817209 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:52:53.817209 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:52:53.817209 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:52:53.817209 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:52:53.817209 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:52:53.817209 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:52:53.819142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:52:53.819142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:52:53.819142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:52:53.819142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:52:53.819142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:52:53.819142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:52:53.819142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:52:53.819142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:52:53.819142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:52:53.819142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:52:53.819142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:52:53.819142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:52:53.819142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:52:53.819142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:52:53.819142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:52:53.819142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:52:53.820019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:52:53.820019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:52:53.820019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:52:53.820019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:52:53.820019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:52:53.820019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:52:53.820019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:52:53.820019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:52:53.820019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:52:53.820019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:52:53.820019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:52:53.820019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:52:53.820019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 83/159] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:52:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:52:53.827032 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e8d6a-f1cb-43d8-9ef3-73015ac384f0 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:53.907737 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:53.964635 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8032b-edbf-4d97-b003-dc1cdb5f2e9f None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:53.965011 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8032b-edbf-4d97-b003-dc1cdb5f2e9f None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:53.965293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8032b-edbf-4d97-b003-dc1cdb5f2e9f None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:53.967162 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 84/160] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:52:53.975189 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40279be-3696-4a75-900a-98713bf33e00 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:53.975553 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40279be-3696-4a75-900a-98713bf33e00 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:53.975915 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40279be-3696-4a75-900a-98713bf33e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:54.000804 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-e40279be-3696-4a75-900a-98713bf33e00 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:54.002101 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e40279be-3696-4a75-900a-98713bf33e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:54.036884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:54.037872 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e8d6a-f1cb-43d8-9ef3-73015ac384f0 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:54.117304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 85/161] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:52:54.144645 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5be69-3afc-459b-aa22-03865bd64c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:54.145401 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5be69-3afc-459b-aa22-03865bd64c27 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:54.145684 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5be69-3afc-459b-aa22-03865bd64c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:54.145959 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5be69-3afc-459b-aa22-03865bd64c27 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:54.146701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5be69-3afc-459b-aa22-03865bd64c27 None admin] RBAC: Adding query filter params (name=member) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:52:54.147006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5be69-3afc-459b-aa22-03865bd64c27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:54.152869 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5be69-3afc-459b-aa22-03865bd64c27 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:54.153628 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40279be-3696-4a75-900a-98713bf33e00 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:54.165018 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 77/162] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 13:52:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:54.187460 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5be69-3afc-459b-aa22-03865bd64c27 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:54.285433 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40279be-3696-4a75-900a-98713bf33e00 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:54.324193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e560320-6f27-4440-bb2f-d94873fa9b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:54.324897 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e560320-6f27-4440-bb2f-d94873fa9b14 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:54.325175 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e560320-6f27-4440-bb2f-d94873fa9b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:54.325432 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e560320-6f27-4440-bb2f-d94873fa9b14 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:54.339332 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e560320-6f27-4440-bb2f-d94873fa9b14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:52:54.345542 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-2e560320-6f27-4440-bb2f-d94873fa9b14 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:52:54.345542 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:52:54.345542 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:52:54.345542 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:52:54.345542 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:52:54.345542 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:52:54.345542 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:52:54.345542 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:52:54.345542 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:52:54.345542 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:52:54.345542 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:52:54.345542 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:52:54.345542 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:52:54.345542 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:52:54.345542 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:52:54.345542 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:52:54.347269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:52:54.347269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:52:54.347269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:52:54.347269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:52:54.347269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:52:54.347269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:52:54.347269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:52:54.347269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:52:54.347269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:52:54.347269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:52:54.347269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:52:54.347269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:52:54.347269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:52:54.347269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:52:54.347269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:52:54.347269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:52:54.348860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:52:54.348860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:52:54.348860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:52:54.348860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:52:54.348860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:52:54.348860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:52:54.348860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:52:54.348860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:52:54.348860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:52:54.348860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:52:54.348860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:52:54.348860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:52:54.348860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 86/163] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 13:52:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:52:54.353935 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e560320-6f27-4440-bb2f-d94873fa9b14 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:54.400462 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:54.492763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5be69-3afc-459b-aa22-03865bd64c27 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:54.516998 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:54.542876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e560320-6f27-4440-bb2f-d94873fa9b14 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:54.614601 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f46d3d-d13c-4b2d-88db-52591d1aa5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:54.615527 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f46d3d-d13c-4b2d-88db-52591d1aa5fe None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:54.615968 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f46d3d-d13c-4b2d-88db-52591d1aa5fe None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:54.616471 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f46d3d-d13c-4b2d-88db-52591d1aa5fe None admin] PATH_INFO: `/v3/users/0cd5be8f747c4089988de317f6307400` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:54.619091 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f46d3d-d13c-4b2d-88db-52591d1aa5fe None admin] RBAC: Authorizing `identity:get_user(user_id=0cd5be8f747c4089988de317f6307400)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:54.621844 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f46d3d-d13c-4b2d-88db-52591d1aa5fe None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:54.632941 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 78/164] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:52:54 2022] GET /identity/v3/users/0cd5be8f747c4089988de317f6307400 => generated 275 bytes in 446 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:52:54.640045 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f46d3d-d13c-4b2d-88db-52591d1aa5fe None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:54.641032 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b4844-03a5-4766-9bfc-df88aba2f0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:54.642267 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b4844-03a5-4766-9bfc-df88aba2f0ae None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:54.643048 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b4844-03a5-4766-9bfc-df88aba2f0ae None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:54.643565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b4844-03a5-4766-9bfc-df88aba2f0ae None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:54.644550 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b4844-03a5-4766-9bfc-df88aba2f0ae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:52:54.645056 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b4844-03a5-4766-9bfc-df88aba2f0ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:54.646780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b4844-03a5-4766-9bfc-df88aba2f0ae None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:54.663756 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 87/165] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:52:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:54.670072 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b4844-03a5-4766-9bfc-df88aba2f0ae None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:54.917871 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f46d3d-d13c-4b2d-88db-52591d1aa5fe None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:54.952775 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:55.024413 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c13f74-211b-48b9-9d33-98ce12b0a25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:55.025374 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c13f74-211b-48b9-9d33-98ce12b0a25a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:55.025770 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c13f74-211b-48b9-9d33-98ce12b0a25a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:55.026076 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c13f74-211b-48b9-9d33-98ce12b0a25a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:55.072181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b4844-03a5-4766-9bfc-df88aba2f0ae None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:55.103618 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:55.184705 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c13f74-211b-48b9-9d33-98ce12b0a25a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:55.186929 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c13f74-211b-48b9-9d33-98ce12b0a25a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:55.226627 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734f78d-ffa5-4bfb-b42a-bc6809f6a488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:55.227706 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734f78d-ffa5-4bfb-b42a-bc6809f6a488 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:55.228199 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734f78d-ffa5-4bfb-b42a-bc6809f6a488 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:55.228727 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734f78d-ffa5-4bfb-b42a-bc6809f6a488 None admin] PATH_INFO: `/v3/users/0cd5be8f747c4089988de317f6307400` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:55.233620 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734f78d-ffa5-4bfb-b42a-bc6809f6a488 None admin] RBAC: Authorizing `identity:get_user(user_id=0cd5be8f747c4089988de317f6307400)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:55.236592 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734f78d-ffa5-4bfb-b42a-bc6809f6a488 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:55.244309 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 88/166] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:52:54 2022] GET /identity/v3/users/0cd5be8f747c4089988de317f6307400 => generated 275 bytes in 570 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:52:55.246751 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df65f3a-6bd2-4153-864f-f6ca1e5d4095 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:55.247270 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df65f3a-6bd2-4153-864f-f6ca1e5d4095 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:55.247749 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df65f3a-6bd2-4153-864f-f6ca1e5d4095 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:55.250395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 89/167] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:55 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:52:55.257944 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df65f3a-6bd2-4153-864f-f6ca1e5d4095 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:55.479195 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 79/168] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:52:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 840 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:52:55.480579 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78263142-3945-4cd6-9f96-a6d74ba075a7 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:55.480952 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78263142-3945-4cd6-9f96-a6d74ba075a7 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:55.481359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78263142-3945-4cd6-9f96-a6d74ba075a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:55.523796 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-78263142-3945-4cd6-9f96-a6d74ba075a7 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:55.525588 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-78263142-3945-4cd6-9f96-a6d74ba075a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:55.628030 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df65f3a-6bd2-4153-864f-f6ca1e5d4095 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:55.711848 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96d0ec-8611-4cf4-a4cd-ea19527c2996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:55.712932 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96d0ec-8611-4cf4-a4cd-ea19527c2996 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:55.713408 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96d0ec-8611-4cf4-a4cd-ea19527c2996 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:55.713873 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96d0ec-8611-4cf4-a4cd-ea19527c2996 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:55.720718 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:55.731819 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 80/169] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:52:55.740751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78263142-3945-4cd6-9f96-a6d74ba075a7 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:55.749780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:55.881396 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a96d0ec-8611-4cf4-a4cd-ea19527c2996 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:55.883402 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a96d0ec-8611-4cf4-a4cd-ea19527c2996 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:56.006767 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78263142-3945-4cd6-9f96-a6d74ba075a7 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:56.090293 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ecfc1-22c5-427d-a2c7-013ecc6d126e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:56.091301 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ecfc1-22c5-427d-a2c7-013ecc6d126e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:56.091766 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ecfc1-22c5-427d-a2c7-013ecc6d126e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:56.092206 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ecfc1-22c5-427d-a2c7-013ecc6d126e None admin] PATH_INFO: `/v3/projects/fd7418cfcedb4164bed000f138751888` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:56.097541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ecfc1-22c5-427d-a2c7-013ecc6d126e None admin] RBAC: Authorizing `identity:get_project(project_id=fd7418cfcedb4164bed000f138751888)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:56.100045 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ecfc1-22c5-427d-a2c7-013ecc6d126e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:56.103137 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 81/170] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:52:55 2022] GET /identity/v3/projects/fd7418cfcedb4164bed000f138751888 => generated 312 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:56.109721 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ecfc1-22c5-427d-a2c7-013ecc6d126e None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:56.121282 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 90/171] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:52:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 870 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:52:56.122174 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96d0ec-8611-4cf4-a4cd-ea19527c2996 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:56.200773 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:56.340082 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96d0ec-8611-4cf4-a4cd-ea19527c2996 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:56.352876 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:56.460600 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ecfc1-22c5-427d-a2c7-013ecc6d126e None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:56.461801 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19bcc7f-5ece-4313-bc90-e655b24e7b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:56.462747 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19bcc7f-5ece-4313-bc90-e655b24e7b5f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:56.463328 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19bcc7f-5ece-4313-bc90-e655b24e7b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:56.463890 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19bcc7f-5ece-4313-bc90-e655b24e7b5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:56.465688 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19bcc7f-5ece-4313-bc90-e655b24e7b5f None admin] RBAC: Adding query filter params (role.id=75aff8cb4cf748a2adbd6c3f141fe8ac, scope.project.id=fd7418cfcedb4164bed000f138751888, user.id=0cd5be8f747c4089988de317f6307400) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:52:56.472539 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19bcc7f-5ece-4313-bc90-e655b24e7b5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:56.476706 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19bcc7f-5ece-4313-bc90-e655b24e7b5f None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:56.499003 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 91/172] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:52:56 2022] GET /identity/v3/role_assignments?user.id=0cd5be8f747c4089988de317f6307400&scope.project.id=fd7418cfcedb4164bed000f138751888&role.id=75aff8cb4cf748a2adbd6c3f141fe8ac => generated 608 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:56.501151 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19bcc7f-5ece-4313-bc90-e655b24e7b5f None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:56.570540 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b272b6-23bb-4499-8ad3-a65b23c4670f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:56.571602 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b272b6-23bb-4499-8ad3-a65b23c4670f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:56.572082 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b272b6-23bb-4499-8ad3-a65b23c4670f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:56.572657 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b272b6-23bb-4499-8ad3-a65b23c4670f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:56.598631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b272b6-23bb-4499-8ad3-a65b23c4670f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:52:56.609897 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-f0b272b6-23bb-4499-8ad3-a65b23c4670f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:52:56.609897 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:52:56.609897 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:52:56.609897 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:52:56.609897 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:52:56.609897 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:52:56.609897 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:52:56.609897 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:52:56.609897 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:52:56.609897 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:52:56.609897 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:52:56.609897 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:52:56.609897 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:52:56.609897 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:52:56.609897 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:52:56.609897 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:52:56.611541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:52:56.611541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:52:56.611541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:52:56.611541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:52:56.611541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:52:56.611541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:52:56.611541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:52:56.611541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:52:56.611541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:52:56.611541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:52:56.611541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:52:56.611541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:52:56.611541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:52:56.611541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:52:56.611541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:52:56.611541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:52:56.613124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:52:56.613124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:52:56.613124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:52:56.613124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:52:56.613124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:52:56.613124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:52:56.613124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:52:56.613124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:52:56.613124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:52:56.613124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:52:56.613124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:52:56.613124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:52:56.618030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 82/173] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:52:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 514 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:52:56.628760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fdb6e-20fa-470b-bb40-04e7dad77ba5 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:56.629799 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fdb6e-20fa-470b-bb40-04e7dad77ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:56.630570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fdb6e-20fa-470b-bb40-04e7dad77ba5 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:56.633394 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 83/174] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:56 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:52:56.635154 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044fdb6e-20fa-470b-bb40-04e7dad77ba5 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:56.793990 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044fdb6e-20fa-470b-bb40-04e7dad77ba5 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:56.836949 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:56.848544 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034325e9-263c-4152-851c-e23212636407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:56.849632 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034325e9-263c-4152-851c-e23212636407 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:56.850122 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034325e9-263c-4152-851c-e23212636407 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:56.850625 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034325e9-263c-4152-851c-e23212636407 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:56.851628 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034325e9-263c-4152-851c-e23212636407 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:52:56.852145 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034325e9-263c-4152-851c-e23212636407 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:56.854584 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034325e9-263c-4152-851c-e23212636407 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:56.866131 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 84/175] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:52:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:56.867829 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc1521-32df-4c2f-9beb-08adebf49940 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:56.868428 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc1521-32df-4c2f-9beb-08adebf49940 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:56.869156 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc1521-32df-4c2f-9beb-08adebf49940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:56.883141 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19bcc7f-5ece-4313-bc90-e655b24e7b5f None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:56.890070 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:56.899802 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-73cc1521-32df-4c2f-9beb-08adebf49940 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:56.902245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-73cc1521-32df-4c2f-9beb-08adebf49940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:56.963588 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8121c59-8e15-4740-beb2-2b4793beaf59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:56.964705 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8121c59-8e15-4740-beb2-2b4793beaf59 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:56.965354 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8121c59-8e15-4740-beb2-2b4793beaf59 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:56.966018 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8121c59-8e15-4740-beb2-2b4793beaf59 None admin] PATH_INFO: `/v3/projects/c18388255f054822b773ec88708292da` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:56.968647 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8121c59-8e15-4740-beb2-2b4793beaf59 None admin] RBAC: Authorizing `identity:get_project(project_id=c18388255f054822b773ec88708292da)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:56.971807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8121c59-8e15-4740-beb2-2b4793beaf59 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:56.975295 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 92/176] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:52:56 2022] GET /identity/v3/projects/c18388255f054822b773ec88708292da => generated 298 bytes in 475 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:56.979148 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19481b40-46bf-4419-abb5-57a4f422e138 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:56.979741 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19481b40-46bf-4419-abb5-57a4f422e138 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:56.980229 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19481b40-46bf-4419-abb5-57a4f422e138 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:56.982073 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 93/177] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:52:56.983962 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19481b40-46bf-4419-abb5-57a4f422e138 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:57.020381 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 85/178] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:52:57.021635 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc1521-32df-4c2f-9beb-08adebf49940 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:57.188738 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19481b40-46bf-4419-abb5-57a4f422e138 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:57.255886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc1521-32df-4c2f-9beb-08adebf49940 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:57.271551 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3585724e-f5c3-4c4f-b536-7c77e93795cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:57.272673 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585724e-f5c3-4c4f-b536-7c77e93795cc None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:57.273167 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585724e-f5c3-4c4f-b536-7c77e93795cc None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:57.273658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585724e-f5c3-4c4f-b536-7c77e93795cc None admin] PATH_INFO: `/v3/users/bf301c0e8be34b7fa129e68f7514a379` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:57.275744 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3585724e-f5c3-4c4f-b536-7c77e93795cc None admin] RBAC: Authorizing `identity:get_user(user_id=bf301c0e8be34b7fa129e68f7514a379)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:57.277282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:57.278632 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3585724e-f5c3-4c4f-b536-7c77e93795cc None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:57.288427 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 94/179] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:52:56 2022] GET /identity/v3/users/bf301c0e8be34b7fa129e68f7514a379 => generated 247 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:57.288427 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6066419f-93be-468d-8859-ee91503c31aa None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:57.288427 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6066419f-93be-468d-8859-ee91503c31aa None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:57.288427 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6066419f-93be-468d-8859-ee91503c31aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:57.299926 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc854ce7-f2e5-466b-b68b-e07b39a16074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:57.300594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc854ce7-f2e5-466b-b68b-e07b39a16074 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:57.300873 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc854ce7-f2e5-466b-b68b-e07b39a16074 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:57.301147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc854ce7-f2e5-466b-b68b-e07b39a16074 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:57.301950 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc854ce7-f2e5-466b-b68b-e07b39a16074 None admin] RBAC: Adding query filter params (role.id=60efb70fa7354ebfb04aa5e9bdad2a78, scope.project.id=c18388255f054822b773ec88708292da, user.id=0cd5be8f747c4089988de317f6307400) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:52:57.302260 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc854ce7-f2e5-466b-b68b-e07b39a16074 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:57.304681 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc854ce7-f2e5-466b-b68b-e07b39a16074 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:57.311934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 86/180] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:52:57 2022] GET /identity/v3/role_assignments?user.id=0cd5be8f747c4089988de317f6307400&scope.project.id=c18388255f054822b773ec88708292da&role.id=60efb70fa7354ebfb04aa5e9bdad2a78 => generated 608 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:57.313480 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76897f8a-3170-4d57-bc97-d208477d5db0 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:57.313766 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76897f8a-3170-4d57-bc97-d208477d5db0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:57.314203 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76897f8a-3170-4d57-bc97-d208477d5db0 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:57.315366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 87/181] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:52:57.316198 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76897f8a-3170-4d57-bc97-d208477d5db0 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:57.318126 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-6066419f-93be-468d-8859-ee91503c31aa None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:57.319821 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-6066419f-93be-468d-8859-ee91503c31aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:57.387697 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:57.449830 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 95/182] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:52:57.451485 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c7019-0824-4353-b258-425f773e0279 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:57.451901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c7019-0824-4353-b258-425f773e0279 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:57.452349 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c7019-0824-4353-b258-425f773e0279 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:57.455139 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 96/183] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:52:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:52:57.462611 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c7019-0824-4353-b258-425f773e0279 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:57.527592 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76897f8a-3170-4d57-bc97-d208477d5db0 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:57.595296 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a6e6b4-7aa7-47bd-ad4e-c9a24431e8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:57.596439 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6e6b4-7aa7-47bd-ad4e-c9a24431e8d6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:57.596854 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6e6b4-7aa7-47bd-ad4e-c9a24431e8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:57.597271 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6e6b4-7aa7-47bd-ad4e-c9a24431e8d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:57.614178 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a6e6b4-7aa7-47bd-ad4e-c9a24431e8d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:52:57.619655 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-22a6e6b4-7aa7-47bd-ad4e-c9a24431e8d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:52:57.619655 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:52:57.619655 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:52:57.619655 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:52:57.619655 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:52:57.619655 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:52:57.619655 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:52:57.619655 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:52:57.619655 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:52:57.619655 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:52:57.619655 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:52:57.619655 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:52:57.619655 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:52:57.619655 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:52:57.619655 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:52:57.619655 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:52:57.621033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:52:57.621033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:52:57.621033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:52:57.621033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:52:57.621033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:52:57.621033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:52:57.621033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:52:57.621033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:52:57.621033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:52:57.621033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:52:57.621033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:52:57.621033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:52:57.621033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:52:57.621033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:52:57.621033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:52:57.621033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:52:57.622496 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:52:57.622496 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:52:57.622496 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:52:57.622496 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:52:57.622496 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:52:57.622496 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:52:57.622496 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:52:57.622496 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:52:57.622496 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:52:57.622496 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:52:57.622496 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:52:57.622496 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:52:57.622496 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 88/184] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:52:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:52:57.622496 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48972e0-4221-401c-9405-b669b749c2d3 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:57.623887 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48972e0-4221-401c-9405-b669b749c2d3 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:57.623887 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48972e0-4221-401c-9405-b669b749c2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:57.643750 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c7019-0824-4353-b258-425f773e0279 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:57.650596 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b48972e0-4221-401c-9405-b669b749c2d3 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:57.652445 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b48972e0-4221-401c-9405-b669b749c2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:57.691104 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13495f77-d692-4ecb-9b50-b6451b2ca27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:57.691741 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13495f77-d692-4ecb-9b50-b6451b2ca27e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:57.692015 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13495f77-d692-4ecb-9b50-b6451b2ca27e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:57.692307 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13495f77-d692-4ecb-9b50-b6451b2ca27e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:57.694304 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:57.736052 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 89/185] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:52:57.737532 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e40324-e8ac-42b4-a1a0-ba474bc0cc5a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:57.737823 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e40324-e8ac-42b4-a1a0-ba474bc0cc5a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:57.738100 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e40324-e8ac-42b4-a1a0-ba474bc0cc5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:57.752598 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:57.758747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-84e40324-e8ac-42b4-a1a0-ba474bc0cc5a None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:52:57.760163 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-84e40324-e8ac-42b4-a1a0-ba474bc0cc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:52:57.791964 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13495f77-d692-4ecb-9b50-b6451b2ca27e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:57.793583 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13495f77-d692-4ecb-9b50-b6451b2ca27e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:57.853926 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 90/186] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:52:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:52:57.862080 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e40324-e8ac-42b4-a1a0-ba474bc0cc5a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:57.935496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 97/187] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:52:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:52:57.937999 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13495f77-d692-4ecb-9b50-b6451b2ca27e None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:58.022129 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e40324-e8ac-42b4-a1a0-ba474bc0cc5a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:58.024053 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:58.037252 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:58.068603 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff03a5d-7138-4c6b-a4cf-dc9d988bf351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:58.069399 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff03a5d-7138-4c6b-a4cf-dc9d988bf351 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:58.069827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff03a5d-7138-4c6b-a4cf-dc9d988bf351 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:58.070205 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff03a5d-7138-4c6b-a4cf-dc9d988bf351 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:58.081801 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff03a5d-7138-4c6b-a4cf-dc9d988bf351 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:52:58.082122 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13495f77-d692-4ecb-9b50-b6451b2ca27e None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:58.088813 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-2ff03a5d-7138-4c6b-a4cf-dc9d988bf351 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:52:58.088813 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:52:58.088813 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:52:58.088813 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:52:58.088813 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:52:58.088813 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:52:58.088813 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:52:58.088813 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:52:58.088813 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:52:58.088813 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:52:58.088813 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:52:58.088813 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:52:58.088813 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:52:58.088813 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:52:58.088813 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:52:58.088813 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:52:58.090519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:52:58.090519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:52:58.090519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:52:58.090519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:52:58.090519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:52:58.090519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:52:58.090519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:52:58.090519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:52:58.090519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:52:58.090519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:52:58.090519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:52:58.090519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:52:58.090519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:52:58.090519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:52:58.090519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:52:58.090519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:52:58.091618 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:52:58.091618 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:52:58.091618 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:52:58.091618 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:52:58.091618 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:52:58.091618 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:52:58.091618 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:52:58.091618 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:52:58.091618 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:52:58.091618 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:52:58.091618 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:52:58.091618 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:52:58.091618 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 91/188] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:52:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:52:58.092882 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff03a5d-7138-4c6b-a4cf-dc9d988bf351 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:58.131294 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff72a44-958e-46c5-81cb-a3cf08765368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:58.131928 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff72a44-958e-46c5-81cb-a3cf08765368 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:58.132210 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff72a44-958e-46c5-81cb-a3cf08765368 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:58.133400 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff72a44-958e-46c5-81cb-a3cf08765368 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:58.133400 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff72a44-958e-46c5-81cb-a3cf08765368 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:52:58.134555 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff72a44-958e-46c5-81cb-a3cf08765368 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:58.134954 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff72a44-958e-46c5-81cb-a3cf08765368 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:58.151185 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 98/189] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:52:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:58.151185 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff72a44-958e-46c5-81cb-a3cf08765368 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:58.250841 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff03a5d-7138-4c6b-a4cf-dc9d988bf351 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:58.292254 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff72a44-958e-46c5-81cb-a3cf08765368 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:58.303386 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7645ffb5-c3f1-460b-b7c6-c4d9ad52e0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:58.303986 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645ffb5-c3f1-460b-b7c6-c4d9ad52e0a3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:58.304466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645ffb5-c3f1-460b-b7c6-c4d9ad52e0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:58.304901 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645ffb5-c3f1-460b-b7c6-c4d9ad52e0a3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:58.322444 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7645ffb5-c3f1-460b-b7c6-c4d9ad52e0a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:52:58.328070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-7645ffb5-c3f1-460b-b7c6-c4d9ad52e0a3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:52:58.328070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:52:58.328070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:52:58.328070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:52:58.328070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:52:58.328070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:52:58.328070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:52:58.328070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:52:58.328070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:52:58.328070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:52:58.328070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:52:58.328070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:52:58.328070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:52:58.328070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:52:58.328070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:52:58.328070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:52:58.329153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:52:58.329153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:52:58.329153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:52:58.329153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:52:58.329153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:52:58.329153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:52:58.329153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:52:58.329153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:52:58.329153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:52:58.329153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:52:58.329153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:52:58.329153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:52:58.329153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:52:58.329153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:52:58.329153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:52:58.329153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:52:58.329951 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:52:58.329951 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:52:58.329951 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:52:58.329951 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:52:58.329951 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:52:58.329951 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:52:58.329951 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:52:58.329951 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:52:58.329951 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:52:58.329951 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:52:58.329951 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:52:58.329951 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:52:58.331423 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:58.331504 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 92/190] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 13:52:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:52:58.331504 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7645ffb5-c3f1-460b-b7c6-c4d9ad52e0a3 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:58.337204 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058f05b-9734-4a60-925b-fdab59d7a587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:58.337920 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058f05b-9734-4a60-925b-fdab59d7a587 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:58.338181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058f05b-9734-4a60-925b-fdab59d7a587 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:58.338449 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058f05b-9734-4a60-925b-fdab59d7a587 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:58.339714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8058f05b-9734-4a60-925b-fdab59d7a587 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:58.341270 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8058f05b-9734-4a60-925b-fdab59d7a587 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:58.343128 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 99/191] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:52:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:58.351646 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058f05b-9734-4a60-925b-fdab59d7a587 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:58.367115 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:58.459262 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7645ffb5-c3f1-460b-b7c6-c4d9ad52e0a3 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:58.470827 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058f05b-9734-4a60-925b-fdab59d7a587 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:58.502726 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29600f2-470a-4d81-909b-48085f1b0f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:58.503383 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29600f2-470a-4d81-909b-48085f1b0f4b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:58.503656 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29600f2-470a-4d81-909b-48085f1b0f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:58.503923 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29600f2-470a-4d81-909b-48085f1b0f4b None admin] PATH_INFO: `/v3/projects/c18388255f054822b773ec88708292da` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:58.505187 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29600f2-470a-4d81-909b-48085f1b0f4b None admin] RBAC: Authorizing `identity:get_project(project_id=c18388255f054822b773ec88708292da)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:58.507007 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29600f2-470a-4d81-909b-48085f1b0f4b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:58.509531 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 93/192] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:52:58 2022] GET /identity/v3/projects/c18388255f054822b773ec88708292da => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:58.510313 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29600f2-470a-4d81-909b-48085f1b0f4b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:58.519357 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75934cf2-a3ce-4c99-9a2b-218e0ff1d1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:58.520031 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75934cf2-a3ce-4c99-9a2b-218e0ff1d1da None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:58.520335 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75934cf2-a3ce-4c99-9a2b-218e0ff1d1da None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:58.520647 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75934cf2-a3ce-4c99-9a2b-218e0ff1d1da None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:58.521384 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75934cf2-a3ce-4c99-9a2b-218e0ff1d1da None admin] RBAC: Adding query filter params (name=reader) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:52:58.521689 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75934cf2-a3ce-4c99-9a2b-218e0ff1d1da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:58.523145 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75934cf2-a3ce-4c99-9a2b-218e0ff1d1da None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:58.534983 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 100/193] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 13:52:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:58.534983 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75934cf2-a3ce-4c99-9a2b-218e0ff1d1da None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:58.620417 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:58.665120 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:58.670558 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75934cf2-a3ce-4c99-9a2b-218e0ff1d1da None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:58.677328 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29600f2-470a-4d81-909b-48085f1b0f4b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:58.707181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7646c0-0627-4d12-b2b5-f303441b4756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:58.707817 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7646c0-0627-4d12-b2b5-f303441b4756 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:58.708092 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7646c0-0627-4d12-b2b5-f303441b4756 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:58.708382 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7646c0-0627-4d12-b2b5-f303441b4756 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:58.709099 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7646c0-0627-4d12-b2b5-f303441b4756 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:52:58.709436 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7646c0-0627-4d12-b2b5-f303441b4756 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:58.710822 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7646c0-0627-4d12-b2b5-f303441b4756 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:58.723865 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc770a-3df3-46e0-9430-2ab10589e25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:58.723865 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc770a-3df3-46e0-9430-2ab10589e25a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:58.723865 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc770a-3df3-46e0-9430-2ab10589e25a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:58.723865 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc770a-3df3-46e0-9430-2ab10589e25a None admin] PATH_INFO: `/v3/users/8dcd8be2726a4f3eaee54e70a9f00394` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:58.724356 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 101/194] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:52:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:58.724356 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7646c0-0627-4d12-b2b5-f303441b4756 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:58.734411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc770a-3df3-46e0-9430-2ab10589e25a None admin] RBAC: Authorizing `identity:get_user(user_id=8dcd8be2726a4f3eaee54e70a9f00394)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:58.735903 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc770a-3df3-46e0-9430-2ab10589e25a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:58.737706 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 94/195] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:52:58 2022] GET /identity/v3/users/8dcd8be2726a4f3eaee54e70a9f00394 => generated 283 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:58.743275 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc770a-3df3-46e0-9430-2ab10589e25a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:58.852487 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7646c0-0627-4d12-b2b5-f303441b4756 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:58.861887 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc770a-3df3-46e0-9430-2ab10589e25a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:58.878048 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:58.891163 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d40b4f-c18e-41cc-aac2-fd3182f6bd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:58.891786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d40b4f-c18e-41cc-aac2-fd3182f6bd45 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:58.892067 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d40b4f-c18e-41cc-aac2-fd3182f6bd45 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:58.892457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d40b4f-c18e-41cc-aac2-fd3182f6bd45 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:58.893428 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d40b4f-c18e-41cc-aac2-fd3182f6bd45 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:58.895063 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d40b4f-c18e-41cc-aac2-fd3182f6bd45 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:58.907701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8067f50-517c-452d-88a1-13e648e67045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:58.908469 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8067f50-517c-452d-88a1-13e648e67045 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:58.908771 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8067f50-517c-452d-88a1-13e648e67045 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:58.909047 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8067f50-517c-452d-88a1-13e648e67045 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:58.909940 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8067f50-517c-452d-88a1-13e648e67045 None admin] RBAC: Adding query filter params (role.id=3e1a9ff016c840c5b51ef7e3da9900ac, scope.project.id=c18388255f054822b773ec88708292da, user.id=bf301c0e8be34b7fa129e68f7514a379) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:52:58.910381 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8067f50-517c-452d-88a1-13e648e67045 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:58.911960 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8067f50-517c-452d-88a1-13e648e67045 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:58.918964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 95/196] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:52:58 2022] GET /identity/v3/role_assignments?user.id=bf301c0e8be34b7fa129e68f7514a379&scope.project.id=c18388255f054822b773ec88708292da&role.id=3e1a9ff016c840c5b51ef7e3da9900ac => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:58.919798 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8067f50-517c-452d-88a1-13e648e67045 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:58.932657 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 102/197] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 13:52:58 2022] POST /identity/v3/users => generated 297 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:52:58.932657 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d40b4f-c18e-41cc-aac2-fd3182f6bd45 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:58.956307 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:59.069032 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8067f50-517c-452d-88a1-13e648e67045 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:59.089861 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d40b4f-c18e-41cc-aac2-fd3182f6bd45 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:59.114850 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c3e5c9-2fae-470d-9c3b-c2f7f5c826a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:59.115434 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3e5c9-2fae-470d-9c3b-c2f7f5c826a4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:59.115707 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3e5c9-2fae-470d-9c3b-c2f7f5c826a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:59.115975 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3e5c9-2fae-470d-9c3b-c2f7f5c826a4 None admin] PATH_INFO: `/v3/users/70ba8bcd60de4443b95ce1e8d22ff2b4` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:59.117613 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c3e5c9-2fae-470d-9c3b-c2f7f5c826a4 None admin] RBAC: Authorizing `identity:get_user(user_id=70ba8bcd60de4443b95ce1e8d22ff2b4)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:59.119169 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c3e5c9-2fae-470d-9c3b-c2f7f5c826a4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:59.120991 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 96/198] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:52:58 2022] GET /identity/v3/users/70ba8bcd60de4443b95ce1e8d22ff2b4 => generated 289 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:59.121889 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c3e5c9-2fae-470d-9c3b-c2f7f5c826a4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:59.143448 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7817d2-dd3d-4c20-98d5-fd5584a914e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:59.144965 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7817d2-dd3d-4c20-98d5-fd5584a914e7 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:59.145669 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7817d2-dd3d-4c20-98d5-fd5584a914e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:59.146185 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7817d2-dd3d-4c20-98d5-fd5584a914e7 None admin] PATH_INFO: `/v3/users/8dcd8be2726a4f3eaee54e70a9f00394` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:59.148485 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7817d2-dd3d-4c20-98d5-fd5584a914e7 None admin] RBAC: Authorizing `identity:get_user(user_id=8dcd8be2726a4f3eaee54e70a9f00394)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:59.151226 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7817d2-dd3d-4c20-98d5-fd5584a914e7 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:59.154243 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 103/199] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:52:58 2022] GET /identity/v3/users/8dcd8be2726a4f3eaee54e70a9f00394 => generated 283 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:59.155919 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7817d2-dd3d-4c20-98d5-fd5584a914e7 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:59.237453 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:59.238865 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:59.266794 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c3e5c9-2fae-470d-9c3b-c2f7f5c826a4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:59.322768 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85a6388-f3be-4aa0-808b-8e694e816752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:59.324978 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a6388-f3be-4aa0-808b-8e694e816752 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:59.325253 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a6388-f3be-4aa0-808b-8e694e816752 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:59.325525 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a6388-f3be-4aa0-808b-8e694e816752 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:59.333387 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7817d2-dd3d-4c20-98d5-fd5584a914e7 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:59.388324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836bf5ad-9d01-456c-9aab-14146e470144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:59.389526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bf5ad-9d01-456c-9aab-14146e470144 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:59.390097 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bf5ad-9d01-456c-9aab-14146e470144 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:59.390703 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bf5ad-9d01-456c-9aab-14146e470144 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:59.417520 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85a6388-f3be-4aa0-808b-8e694e816752 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:59.419201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85a6388-f3be-4aa0-808b-8e694e816752 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:59.527826 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836bf5ad-9d01-456c-9aab-14146e470144 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:59.530579 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836bf5ad-9d01-456c-9aab-14146e470144 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:59.542795 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:59.584042 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 97/200] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:52:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:52:59.585705 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85a6388-f3be-4aa0-808b-8e694e816752 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:59.614625 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:59.692388 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 104/201] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:52:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:52:59.692388 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836bf5ad-9d01-456c-9aab-14146e470144 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:59.737654 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85a6388-f3be-4aa0-808b-8e694e816752 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:59.789711 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e8209-1d85-46a3-8959-b97755a1f20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:59.791075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e8209-1d85-46a3-8959-b97755a1f20c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:59.791764 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e8209-1d85-46a3-8959-b97755a1f20c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:59.792472 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e8209-1d85-46a3-8959-b97755a1f20c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:59.825416 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836bf5ad-9d01-456c-9aab-14146e470144 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:52:59.872062 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:52:59.879416 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f50a3e-eba4-48aa-a131-ee6ec9de688c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:52:59.880830 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f50a3e-eba4-48aa-a131-ee6ec9de688c None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:52:59.881486 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f50a3e-eba4-48aa-a131-ee6ec9de688c None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:52:59.882198 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f50a3e-eba4-48aa-a131-ee6ec9de688c None admin] PATH_INFO: `/v3/projects/7489ab03e4a8475cab57be63c14ec0c0` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:52:59.884738 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f50a3e-eba4-48aa-a131-ee6ec9de688c None admin] RBAC: Authorizing `identity:get_project(project_id=7489ab03e4a8475cab57be63c14ec0c0)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:59.887270 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f50a3e-eba4-48aa-a131-ee6ec9de688c None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:59.889578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 105/202] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:52:59 2022] GET /identity/v3/projects/7489ab03e4a8475cab57be63c14ec0c0 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:52:59.891426 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f50a3e-eba4-48aa-a131-ee6ec9de688c None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:52:59.895281 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e8209-1d85-46a3-8959-b97755a1f20c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:52:59.897297 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e8209-1d85-46a3-8959-b97755a1f20c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:52:59.922976 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:00.043853 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 98/203] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:52:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:53:00.049215 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e8209-1d85-46a3-8959-b97755a1f20c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:00.055468 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f50a3e-eba4-48aa-a131-ee6ec9de688c None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:00.105659 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71a8f6-91aa-48e7-8dce-39fb2f38e9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:00.106540 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71a8f6-91aa-48e7-8dce-39fb2f38e9c7 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:00.107016 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71a8f6-91aa-48e7-8dce-39fb2f38e9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:00.107357 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71a8f6-91aa-48e7-8dce-39fb2f38e9c7 None admin] PATH_INFO: `/v3/projects/c18388255f054822b773ec88708292da` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:00.108704 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd71a8f6-91aa-48e7-8dce-39fb2f38e9c7 None admin] RBAC: Authorizing `identity:get_project(project_id=c18388255f054822b773ec88708292da)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:00.110764 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd71a8f6-91aa-48e7-8dce-39fb2f38e9c7 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:00.113557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 106/204] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:52:59 2022] GET /identity/v3/projects/c18388255f054822b773ec88708292da => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:00.117810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71a8f6-91aa-48e7-8dce-39fb2f38e9c7 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:00.180501 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:00.209751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e8209-1d85-46a3-8959-b97755a1f20c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:00.232818 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:00.252207 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71a8f6-91aa-48e7-8dce-39fb2f38e9c7 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:00.259320 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f218e2-2d8d-4965-95b5-3aa6f20968cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:00.260259 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f218e2-2d8d-4965-95b5-3aa6f20968cf None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:00.260710 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f218e2-2d8d-4965-95b5-3aa6f20968cf None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:00.261140 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f218e2-2d8d-4965-95b5-3aa6f20968cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:00.262761 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f218e2-2d8d-4965-95b5-3aa6f20968cf None admin] RBAC: Adding query filter params (role.id=3e1a9ff016c840c5b51ef7e3da9900ac, scope.project.id=7489ab03e4a8475cab57be63c14ec0c0, user.id=8dcd8be2726a4f3eaee54e70a9f00394) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:00.263244 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f218e2-2d8d-4965-95b5-3aa6f20968cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:00.265553 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f218e2-2d8d-4965-95b5-3aa6f20968cf None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:00.275447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 99/205] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:53:00 2022] GET /identity/v3/role_assignments?user.id=8dcd8be2726a4f3eaee54e70a9f00394&scope.project.id=7489ab03e4a8475cab57be63c14ec0c0&role.id=3e1a9ff016c840c5b51ef7e3da9900ac => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:00.276527 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f218e2-2d8d-4965-95b5-3aa6f20968cf None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:00.296352 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b4df7-4ae2-4ce0-a38f-d07a2d276481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:00.297359 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b4df7-4ae2-4ce0-a38f-d07a2d276481 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:00.297639 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b4df7-4ae2-4ce0-a38f-d07a2d276481 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:00.298027 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b4df7-4ae2-4ce0-a38f-d07a2d276481 None admin] PATH_INFO: `/v3/projects/7489ab03e4a8475cab57be63c14ec0c0` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:00.299525 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719b4df7-4ae2-4ce0-a38f-d07a2d276481 None admin] RBAC: Authorizing `identity:get_project(project_id=7489ab03e4a8475cab57be63c14ec0c0)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:00.301375 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719b4df7-4ae2-4ce0-a38f-d07a2d276481 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:00.303447 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 107/206] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:53:00 2022] GET /identity/v3/projects/7489ab03e4a8475cab57be63c14ec0c0 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:00.309734 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b4df7-4ae2-4ce0-a38f-d07a2d276481 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:00.421238 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f218e2-2d8d-4965-95b5-3aa6f20968cf None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:00.478934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f9879-ab03-403f-9912-f211a22397f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:00.480144 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f9879-ab03-403f-9912-f211a22397f0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:00.481935 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f9879-ab03-403f-9912-f211a22397f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:00.483770 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f9879-ab03-403f-9912-f211a22397f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:00.484941 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f9879-ab03-403f-9912-f211a22397f0 None admin] RBAC: Adding query filter params (role.id=9db7d52dd9514261a5a52591fbf1feef, scope.project.id=c18388255f054822b773ec88708292da, user.id=70ba8bcd60de4443b95ce1e8d22ff2b4) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:00.485674 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f9879-ab03-403f-9912-f211a22397f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:00.488106 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f9879-ab03-403f-9912-f211a22397f0 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:00.503554 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 100/207] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:53:00 2022] GET /identity/v3/role_assignments?user.id=70ba8bcd60de4443b95ce1e8d22ff2b4&scope.project.id=c18388255f054822b773ec88708292da&role.id=9db7d52dd9514261a5a52591fbf1feef => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:00.552422 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b4df7-4ae2-4ce0-a38f-d07a2d276481 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:00.619038 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3422fc3-0146-42ff-bdcf-c1e2b758ed82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:00.619702 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3422fc3-0146-42ff-bdcf-c1e2b758ed82 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:00.619987 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3422fc3-0146-42ff-bdcf-c1e2b758ed82 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:00.620260 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3422fc3-0146-42ff-bdcf-c1e2b758ed82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:00.621107 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3422fc3-0146-42ff-bdcf-c1e2b758ed82 None admin] RBAC: Adding query filter params (role.id=60efb70fa7354ebfb04aa5e9bdad2a78, scope.project.id=7489ab03e4a8475cab57be63c14ec0c0, user.id=8dcd8be2726a4f3eaee54e70a9f00394) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:00.621425 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3422fc3-0146-42ff-bdcf-c1e2b758ed82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:00.622948 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3422fc3-0146-42ff-bdcf-c1e2b758ed82 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:00.630049 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 108/208] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:53:00 2022] GET /identity/v3/role_assignments?user.id=8dcd8be2726a4f3eaee54e70a9f00394&scope.project.id=7489ab03e4a8475cab57be63c14ec0c0&role.id=60efb70fa7354ebfb04aa5e9bdad2a78 => generated 259 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:53:01.765408 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b24d5e-0b69-4ec5-aaef-99b22fc3a844 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:01.765954 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b24d5e-0b69-4ec5-aaef-99b22fc3a844 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:01.765954 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b24d5e-0b69-4ec5-aaef-99b22fc3a844 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:01.767317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 101/209] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:53:01.774351 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb87c5-faac-4892-ad60-2da732064647 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:01.774892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb87c5-faac-4892-ad60-2da732064647 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:01.775322 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb87c5-faac-4892-ad60-2da732064647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:01.802902 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39357ac3-21f9-444f-ace7-22029cde8586 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:01.803186 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39357ac3-21f9-444f-ace7-22029cde8586 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:01.803376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39357ac3-21f9-444f-ace7-22029cde8586 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:01.804800 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 102/210] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:53:01.807305 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-47cb87c5-faac-4892-ad60-2da732064647 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:01.809490 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-47cb87c5-faac-4892-ad60-2da732064647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:01.810521 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ceb01-cb22-41e6-a232-08430bf7fbfb None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:01.810808 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ceb01-cb22-41e6-a232-08430bf7fbfb None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:01.811087 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ceb01-cb22-41e6-a232-08430bf7fbfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:01.830852 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a37ceb01-cb22-41e6-a232-08430bf7fbfb None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:01.832126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a37ceb01-cb22-41e6-a232-08430bf7fbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:01.832446 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:01.834539 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:01.920342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 103/211] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:01.940047 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ceb01-cb22-41e6-a232-08430bf7fbfb None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:01.948975 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 109/212] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:01.975563 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb87c5-faac-4892-ad60-2da732064647 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:02.084181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb87c5-faac-4892-ad60-2da732064647 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:02.120721 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ceb01-cb22-41e6-a232-08430bf7fbfb None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:02.134898 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7691bae9-f5bb-48a3-88c5-120d11af0ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:02.135687 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691bae9-f5bb-48a3-88c5-120d11af0ebf None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:02.136031 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691bae9-f5bb-48a3-88c5-120d11af0ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:02.136401 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691bae9-f5bb-48a3-88c5-120d11af0ebf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:02.137987 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691bae9-f5bb-48a3-88c5-120d11af0ebf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:02.139391 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691bae9-f5bb-48a3-88c5-120d11af0ebf None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:02.149328 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 110/213] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:53:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:53:02.150420 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7691bae9-f5bb-48a3-88c5-120d11af0ebf None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:02.172233 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:02.194529 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913df12d-73c7-4c27-949d-09fa0545e8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:02.196219 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913df12d-73c7-4c27-949d-09fa0545e8c0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:02.197003 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913df12d-73c7-4c27-949d-09fa0545e8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:02.197770 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913df12d-73c7-4c27-949d-09fa0545e8c0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:02.211846 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:02.219942 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913df12d-73c7-4c27-949d-09fa0545e8c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:02.231415 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-913df12d-73c7-4c27-949d-09fa0545e8c0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:53:02.231415 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:02.231415 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:02.231415 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:02.231415 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:02.231415 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:02.231415 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:02.231415 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:02.231415 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:02.231415 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:02.231415 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:02.231415 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:02.231415 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:02.231415 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:02.231415 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:02.231415 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:02.233124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:02.233124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:02.233124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:02.233124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:02.233124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:02.233124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:02.233124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:02.233124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:02.233124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:02.233124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:02.233124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:02.233124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:02.233124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:02.233124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:02.233124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:02.233124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:02.234661 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:02.234661 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:02.234661 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:02.234661 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:02.234661 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:02.234661 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:02.234661 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:02.234661 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:02.234661 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:02.234661 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:02.234661 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:53:02.234661 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:53:02.238937 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 104/214] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:53:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:53:02.245092 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913df12d-73c7-4c27-949d-09fa0545e8c0 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:02.299815 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7691bae9-f5bb-48a3-88c5-120d11af0ebf None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:02.336075 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61176d-b5f4-4ce7-8a77-ca4db93139d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:02.336781 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61176d-b5f4-4ce7-8a77-ca4db93139d0 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:02.337066 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61176d-b5f4-4ce7-8a77-ca4db93139d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:02.337344 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61176d-b5f4-4ce7-8a77-ca4db93139d0 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:02.338043 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf61176d-b5f4-4ce7-8a77-ca4db93139d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:02.339573 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf61176d-b5f4-4ce7-8a77-ca4db93139d0 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:02.363440 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 111/215] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 13:53:02 2022] POST /identity/v3/users => generated 297 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:53:02.434467 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913df12d-73c7-4c27-949d-09fa0545e8c0 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:02.510393 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b0bf9e-b6d5-4e3c-96f8-35a61c157599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:02.511958 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0bf9e-b6d5-4e3c-96f8-35a61c157599 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:02.512786 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0bf9e-b6d5-4e3c-96f8-35a61c157599 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:02.513538 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0bf9e-b6d5-4e3c-96f8-35a61c157599 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:02.515181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b0bf9e-b6d5-4e3c-96f8-35a61c157599 None admin] RBAC: Adding query filter params (name=member) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:02.516150 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b0bf9e-b6d5-4e3c-96f8-35a61c157599 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:02.519385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b0bf9e-b6d5-4e3c-96f8-35a61c157599 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:02.535891 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 105/216] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 13:53:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:53:02.546135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61176d-b5f4-4ce7-8a77-ca4db93139d0 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:02.635121 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:02.671095 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61176d-b5f4-4ce7-8a77-ca4db93139d0 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:02.709455 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56edac64-a6d6-412d-ad73-17214d97c3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:02.710259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56edac64-a6d6-412d-ad73-17214d97c3e2 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:02.710543 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56edac64-a6d6-412d-ad73-17214d97c3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:02.710819 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56edac64-a6d6-412d-ad73-17214d97c3e2 None admin] PATH_INFO: `/v3/users/7822533aa26c4490a849a4da2666c8f5` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:02.727132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56edac64-a6d6-412d-ad73-17214d97c3e2 None admin] RBAC: Authorizing `identity:get_user(user_id=7822533aa26c4490a849a4da2666c8f5)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:02.729010 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56edac64-a6d6-412d-ad73-17214d97c3e2 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:02.731358 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 112/217] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:53:02 2022] GET /identity/v3/users/7822533aa26c4490a849a4da2666c8f5 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:02.737184 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b0bf9e-b6d5-4e3c-96f8-35a61c157599 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:02.784345 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1f9a0-6910-4dad-91dd-0a4ecf6bf7bc None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:02.785307 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1f9a0-6910-4dad-91dd-0a4ecf6bf7bc None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:02.785753 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1f9a0-6910-4dad-91dd-0a4ecf6bf7bc None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:02.788410 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 113/218] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:02.793892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26fc93-1ac2-477e-a5e8-f0d188bb5bf0 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:02.794577 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26fc93-1ac2-477e-a5e8-f0d188bb5bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:02.795086 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26fc93-1ac2-477e-a5e8-f0d188bb5bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:02.814843 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:02.818125 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-cb26fc93-1ac2-477e-a5e8-f0d188bb5bf0 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:02.819348 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-cb26fc93-1ac2-477e-a5e8-f0d188bb5bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:02.862869 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b0bf9e-b6d5-4e3c-96f8-35a61c157599 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:02.906984 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5419fd4f-2931-4c24-9e1b-11b3e27d1a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:02.908013 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5419fd4f-2931-4c24-9e1b-11b3e27d1a35 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:02.908377 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5419fd4f-2931-4c24-9e1b-11b3e27d1a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:02.908657 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5419fd4f-2931-4c24-9e1b-11b3e27d1a35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:02.914027 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 114/219] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:02.961141 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26fc93-1ac2-477e-a5e8-f0d188bb5bf0 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:03.033981 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:03.062420 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5419fd4f-2931-4c24-9e1b-11b3e27d1a35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:03.064102 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5419fd4f-2931-4c24-9e1b-11b3e27d1a35 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:03.105816 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26fc93-1ac2-477e-a5e8-f0d188bb5bf0 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:03.155234 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:03.197053 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a136b7-70e3-4ac1-832e-fe87c0642187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:03.197946 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a136b7-70e3-4ac1-832e-fe87c0642187 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:03.198361 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a136b7-70e3-4ac1-832e-fe87c0642187 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:03.198769 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a136b7-70e3-4ac1-832e-fe87c0642187 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:03.234820 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a136b7-70e3-4ac1-832e-fe87c0642187 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:03.243199 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-66a136b7-70e3-4ac1-832e-fe87c0642187 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:53:03.243199 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:03.243199 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:03.243199 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:03.243199 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:03.243199 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:03.243199 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:03.243199 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:03.243199 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:03.243199 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:03.243199 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:03.243199 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:03.243199 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:03.243199 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:03.243199 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:03.243199 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:03.244137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:03.244137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:03.244137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:03.244137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:03.244137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:03.244137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:03.244137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:03.244137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:03.244137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:03.244137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:03.244137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:03.244137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:03.244137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:03.244137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:03.244137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:03.244137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:03.244999 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:03.244999 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:03.244999 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:03.244999 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:03.244999 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:03.244999 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:03.244999 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:03.244999 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:03.244999 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:03.244999 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:03.244999 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:53:03.244999 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:03.257933 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 106/220] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:53:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:53:03.258810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 115/221] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:53:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:53:03.265134 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5419fd4f-2931-4c24-9e1b-11b3e27d1a35 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:03.265965 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a136b7-70e3-4ac1-832e-fe87c0642187 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:03.436646 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5419fd4f-2931-4c24-9e1b-11b3e27d1a35 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:03.495558 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970a771-25d5-43be-a542-31c4456dc03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:03.496721 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970a771-25d5-43be-a542-31c4456dc03a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:03.497220 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970a771-25d5-43be-a542-31c4456dc03a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:03.497727 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970a771-25d5-43be-a542-31c4456dc03a None admin] PATH_INFO: `/v3/projects/7489ab03e4a8475cab57be63c14ec0c0` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:03.502990 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970a771-25d5-43be-a542-31c4456dc03a None admin] RBAC: Authorizing `identity:get_project(project_id=7489ab03e4a8475cab57be63c14ec0c0)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:03.505862 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970a771-25d5-43be-a542-31c4456dc03a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:03.510306 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 107/222] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:53:03 2022] GET /identity/v3/projects/7489ab03e4a8475cab57be63c14ec0c0 => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:03.519848 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a136b7-70e3-4ac1-832e-fe87c0642187 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:03.520707 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970a771-25d5-43be-a542-31c4456dc03a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:03.548030 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:03.569185 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a636a-5c5c-4cac-baaa-e9bead54f458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:03.569923 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a636a-5c5c-4cac-baaa-e9bead54f458 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:03.570249 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a636a-5c5c-4cac-baaa-e9bead54f458 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:03.570481 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a636a-5c5c-4cac-baaa-e9bead54f458 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:03.571233 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a636a-5c5c-4cac-baaa-e9bead54f458 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:03.571539 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a636a-5c5c-4cac-baaa-e9bead54f458 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:03.573053 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a636a-5c5c-4cac-baaa-e9bead54f458 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:03.584422 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 116/223] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:53:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:03.585936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc29801-d795-4ece-9689-8a1f9ff42cea None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:03.586218 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc29801-d795-4ece-9689-8a1f9ff42cea None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:03.586490 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc29801-d795-4ece-9689-8a1f9ff42cea None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:03.593325 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 117/224] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:03 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:03.594826 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc29801-d795-4ece-9689-8a1f9ff42cea None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:03.609760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:03.721013 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc29801-d795-4ece-9689-8a1f9ff42cea None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:03.746619 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970a771-25d5-43be-a542-31c4456dc03a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:03.759955 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dff498-61ed-4867-8d9b-3f98d53bc4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:03.760834 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dff498-61ed-4867-8d9b-3f98d53bc4c4 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:03.761331 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dff498-61ed-4867-8d9b-3f98d53bc4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:03.761662 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dff498-61ed-4867-8d9b-3f98d53bc4c4 None admin] PATH_INFO: `/v3/users/8dcd8be2726a4f3eaee54e70a9f00394` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:03.762876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dff498-61ed-4867-8d9b-3f98d53bc4c4 None admin] RBAC: Authorizing `identity:get_user(user_id=8dcd8be2726a4f3eaee54e70a9f00394)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:03.764647 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dff498-61ed-4867-8d9b-3f98d53bc4c4 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:03.766522 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 118/225] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:53:03 2022] GET /identity/v3/users/8dcd8be2726a4f3eaee54e70a9f00394 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:53:03.768030 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d762e52-9f21-4806-afc8-9c753d2bced8 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:03.772746 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d762e52-9f21-4806-afc8-9c753d2bced8 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:03.773082 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d762e52-9f21-4806-afc8-9c753d2bced8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:03.796002 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-0d762e52-9f21-4806-afc8-9c753d2bced8 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:03.797676 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0d762e52-9f21-4806-afc8-9c753d2bced8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:03.813157 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28dee8c-a1ca-40fc-8b44-ac59b6e65cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:03.814316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28dee8c-a1ca-40fc-8b44-ac59b6e65cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:03.814790 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28dee8c-a1ca-40fc-8b44-ac59b6e65cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:03.815346 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28dee8c-a1ca-40fc-8b44-ac59b6e65cf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:03.816977 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28dee8c-a1ca-40fc-8b44-ac59b6e65cf0 None admin] RBAC: Adding query filter params (role.id=75aff8cb4cf748a2adbd6c3f141fe8ac, scope.project.id=7489ab03e4a8475cab57be63c14ec0c0, user.id=7822533aa26c4490a849a4da2666c8f5) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:03.817612 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28dee8c-a1ca-40fc-8b44-ac59b6e65cf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:03.820484 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28dee8c-a1ca-40fc-8b44-ac59b6e65cf0 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:03.831408 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 108/226] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:53:03 2022] GET /identity/v3/role_assignments?user.id=7822533aa26c4490a849a4da2666c8f5&scope.project.id=7489ab03e4a8475cab57be63c14ec0c0&role.id=75aff8cb4cf748a2adbd6c3f141fe8ac => generated 259 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:53:03.841208 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28dee8c-a1ca-40fc-8b44-ac59b6e65cf0 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:03.851206 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:03.882715 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 119/227] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:03.915975 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d762e52-9f21-4806-afc8-9c753d2bced8 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:04.006520 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:04.026006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28dee8c-a1ca-40fc-8b44-ac59b6e65cf0 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:04.121388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652af7df-a7d5-4ac2-8d2a-ee2747ed14ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:04.122823 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652af7df-a7d5-4ac2-8d2a-ee2747ed14ca None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:04.123497 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652af7df-a7d5-4ac2-8d2a-ee2747ed14ca None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:04.124171 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652af7df-a7d5-4ac2-8d2a-ee2747ed14ca None admin] PATH_INFO: `/v3/projects/7489ab03e4a8475cab57be63c14ec0c0` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:04.129569 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652af7df-a7d5-4ac2-8d2a-ee2747ed14ca None admin] RBAC: Authorizing `identity:get_project(project_id=7489ab03e4a8475cab57be63c14ec0c0)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:04.132381 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d762e52-9f21-4806-afc8-9c753d2bced8 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:04.132896 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652af7df-a7d5-4ac2-8d2a-ee2747ed14ca None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:04.135999 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 109/228] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:53:03 2022] GET /identity/v3/projects/7489ab03e4a8475cab57be63c14ec0c0 => generated 302 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:04.143986 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652af7df-a7d5-4ac2-8d2a-ee2747ed14ca None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:04.196938 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267f155d-341f-4e36-bfa7-994cc1837cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:04.197632 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f155d-341f-4e36-bfa7-994cc1837cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:04.197913 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f155d-341f-4e36-bfa7-994cc1837cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:04.198187 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f155d-341f-4e36-bfa7-994cc1837cf1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:04.211685 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267f155d-341f-4e36-bfa7-994cc1837cf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:04.216929 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-267f155d-341f-4e36-bfa7-994cc1837cf1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:53:04.216929 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:04.216929 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:04.216929 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:04.216929 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:04.216929 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:04.216929 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:04.216929 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:04.216929 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:04.216929 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:04.216929 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:04.216929 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:04.216929 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:04.216929 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:04.216929 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:04.216929 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:04.217892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:04.217892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:04.217892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:04.217892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:04.217892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:04.217892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:04.217892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:04.217892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:04.217892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:04.217892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:04.217892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:04.217892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:04.217892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:04.217892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:04.217892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:04.217892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:04.218743 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:04.218743 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:04.218743 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:04.218743 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:04.218743 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:04.218743 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:04.218743 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:04.218743 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:04.218743 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:04.218743 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:04.218743 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:53:04.218743 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:04.218743 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 120/229] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 13:53:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:53:04.228700 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267f155d-341f-4e36-bfa7-994cc1837cf1 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:04.334280 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652af7df-a7d5-4ac2-8d2a-ee2747ed14ca None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:04.375703 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:04.394157 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f77edb-cfdf-4bf2-985d-f452533b2001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:04.394850 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f77edb-cfdf-4bf2-985d-f452533b2001 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:04.395129 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f77edb-cfdf-4bf2-985d-f452533b2001 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:04.395410 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f77edb-cfdf-4bf2-985d-f452533b2001 None admin] PATH_INFO: `/v3/projects/7489ab03e4a8475cab57be63c14ec0c0/users/8dcd8be2726a4f3eaee54e70a9f00394/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:04.399034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f77edb-cfdf-4bf2-985d-f452533b2001 None admin] RBAC: Authorizing `identity:create_grant(project_id=7489ab03e4a8475cab57be63c14ec0c0, user_id=8dcd8be2726a4f3eaee54e70a9f00394, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:04.401584 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f77edb-cfdf-4bf2-985d-f452533b2001 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:04.427705 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 110/230] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 13:53:04 2022] PUT /identity/v3/projects/7489ab03e4a8475cab57be63c14ec0c0/users/8dcd8be2726a4f3eaee54e70a9f00394/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:53:04.485585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267f155d-341f-4e36-bfa7-994cc1837cf1 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:04.549611 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83f8f2-21c0-4f03-993d-772d27ffff27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:04.550722 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83f8f2-21c0-4f03-993d-772d27ffff27 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:04.551199 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83f8f2-21c0-4f03-993d-772d27ffff27 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:04.551665 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83f8f2-21c0-4f03-993d-772d27ffff27 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:04.553103 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df83f8f2-21c0-4f03-993d-772d27ffff27 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:04.553687 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df83f8f2-21c0-4f03-993d-772d27ffff27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:04.556391 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df83f8f2-21c0-4f03-993d-772d27ffff27 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:04.573712 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 121/231] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:53:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:04.583213 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f77edb-cfdf-4bf2-985d-f452533b2001 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:04.630064 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:04.798551 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f77edb-cfdf-4bf2-985d-f452533b2001 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:04.861391 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b8d414-7178-4d38-8f4b-2d4ca4720553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:04.862641 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b8d414-7178-4d38-8f4b-2d4ca4720553 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:04.862990 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b8d414-7178-4d38-8f4b-2d4ca4720553 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:04.863443 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b8d414-7178-4d38-8f4b-2d4ca4720553 None admin] PATH_INFO: `/v3/users/8dcd8be2726a4f3eaee54e70a9f00394` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:04.865728 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b8d414-7178-4d38-8f4b-2d4ca4720553 None admin] RBAC: Authorizing `identity:get_user(user_id=8dcd8be2726a4f3eaee54e70a9f00394)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:04.868549 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b8d414-7178-4d38-8f4b-2d4ca4720553 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:04.872108 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 111/232] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:53:04 2022] GET /identity/v3/users/8dcd8be2726a4f3eaee54e70a9f00394 => generated 283 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:04.881417 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83f8f2-21c0-4f03-993d-772d27ffff27 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:05.088071 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:05.094994 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83f8f2-21c0-4f03-993d-772d27ffff27 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:05.159931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e38dc33-98e9-4837-b9b3-250390798bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:05.161547 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38dc33-98e9-4837-b9b3-250390798bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:05.162308 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38dc33-98e9-4837-b9b3-250390798bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:05.163044 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38dc33-98e9-4837-b9b3-250390798bf0 None admin] PATH_INFO: `/v3/projects/7489ab03e4a8475cab57be63c14ec0c0` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:05.165535 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e38dc33-98e9-4837-b9b3-250390798bf0 None admin] RBAC: Authorizing `identity:get_project(project_id=7489ab03e4a8475cab57be63c14ec0c0)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:05.168635 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e38dc33-98e9-4837-b9b3-250390798bf0 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:05.172850 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 122/233] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:53:04 2022] GET /identity/v3/projects/7489ab03e4a8475cab57be63c14ec0c0 => generated 302 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:05.182523 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b8d414-7178-4d38-8f4b-2d4ca4720553 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:05.361298 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:05.389214 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b8d414-7178-4d38-8f4b-2d4ca4720553 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:05.468166 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a1c05-d9e2-494b-8c0b-c7238f02c7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:05.470012 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a1c05-d9e2-494b-8c0b-c7238f02c7c7 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:05.470962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a1c05-d9e2-494b-8c0b-c7238f02c7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:05.471701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a1c05-d9e2-494b-8c0b-c7238f02c7c7 None admin] PATH_INFO: `/v3/projects/7489ab03e4a8475cab57be63c14ec0c0/users/8dcd8be2726a4f3eaee54e70a9f00394/roles/60efb70fa7354ebfb04aa5e9bdad2a78` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:05.475820 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a1c05-d9e2-494b-8c0b-c7238f02c7c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=7489ab03e4a8475cab57be63c14ec0c0, user_id=8dcd8be2726a4f3eaee54e70a9f00394, role_id=60efb70fa7354ebfb04aa5e9bdad2a78)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:05.483206 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a1c05-d9e2-494b-8c0b-c7238f02c7c7 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:05.518852 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 112/234] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 13:53:05 2022] PUT /identity/v3/projects/7489ab03e4a8475cab57be63c14ec0c0/users/8dcd8be2726a4f3eaee54e70a9f00394/roles/60efb70fa7354ebfb04aa5e9bdad2a78 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:53:05.595507 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ada7c2-df08-42e8-90f3-7d43ea7ef8ca None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:05.596631 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ada7c2-df08-42e8-90f3-7d43ea7ef8ca None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:05.597372 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ada7c2-df08-42e8-90f3-7d43ea7ef8ca None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:05.600221 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 123/235] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:05.606932 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd236962-f1c0-4151-a0d7-9ea68a8f1535 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:05.607692 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd236962-f1c0-4151-a0d7-9ea68a8f1535 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:05.608390 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd236962-f1c0-4151-a0d7-9ea68a8f1535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:05.641126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-fd236962-f1c0-4151-a0d7-9ea68a8f1535 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:05.643719 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fd236962-f1c0-4151-a0d7-9ea68a8f1535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:05.753889 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 113/236] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:05.803587 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ada7c2-df08-42e8-90f3-7d43ea7ef8ca None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:05.853204 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a92700-adaf-473d-a72b-9434d6816d27 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:05.853616 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a92700-adaf-473d-a72b-9434d6816d27 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:05.854021 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a92700-adaf-473d-a72b-9434d6816d27 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:05.856402 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 114/237] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:05.863909 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a0e97-1148-4b4d-b2eb-de5d01bdaa0b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:05.864570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a0e97-1148-4b4d-b2eb-de5d01bdaa0b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:05.864785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a0e97-1148-4b4d-b2eb-de5d01bdaa0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:05.891048 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a41a0e97-1148-4b4d-b2eb-de5d01bdaa0b None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:05.893050 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a41a0e97-1148-4b4d-b2eb-de5d01bdaa0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:05.973880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ada7c2-df08-42e8-90f3-7d43ea7ef8ca None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:06.005083 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 115/238] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:53:06.031919 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a0e97-1148-4b4d-b2eb-de5d01bdaa0b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:06.043608 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c58a7-e450-419e-92c4-94334f1e6f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:06.044319 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c58a7-e450-419e-92c4-94334f1e6f8d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:06.044589 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c58a7-e450-419e-92c4-94334f1e6f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:06.044864 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c58a7-e450-419e-92c4-94334f1e6f8d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:06.048666 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:06.063772 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72c58a7-e450-419e-92c4-94334f1e6f8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:06.078304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-f72c58a7-e450-419e-92c4-94334f1e6f8d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:53:06.078304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:06.078304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:06.078304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:06.078304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:06.078304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:06.078304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:06.078304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:06.078304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:06.078304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:06.078304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:06.078304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:06.078304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:06.078304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:06.078304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:06.078304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:06.079497 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:06.079497 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:06.079497 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:06.079497 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:06.079497 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:06.079497 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:06.079497 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:06.079497 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:06.079497 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:06.079497 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:06.079497 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:06.079497 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:06.079497 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:06.079497 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:06.079497 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:06.079497 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:06.080806 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:06.080806 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:06.080806 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:06.080806 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:06.080806 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:06.080806 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:06.080806 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:06.080806 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:06.080806 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:06.080806 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:06.080806 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:53:06.080806 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:06.080806 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 124/239] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:53:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:06.086421 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c58a7-e450-419e-92c4-94334f1e6f8d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:06.152476 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:06.179627 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a0e97-1148-4b4d-b2eb-de5d01bdaa0b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:06.225094 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcfb2c5-d058-4683-823e-05fce4f6f2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:06.226201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcfb2c5-d058-4683-823e-05fce4f6f2a0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:06.226702 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcfb2c5-d058-4683-823e-05fce4f6f2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:06.227197 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcfb2c5-d058-4683-823e-05fce4f6f2a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:06.228874 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcfb2c5-d058-4683-823e-05fce4f6f2a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:06.230655 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcfb2c5-d058-4683-823e-05fce4f6f2a0 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:06.232612 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 116/240] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:53:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:06.238536 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcfb2c5-d058-4683-823e-05fce4f6f2a0 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:06.348970 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:06.349696 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c58a7-e450-419e-92c4-94334f1e6f8d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:06.380551 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcfb2c5-d058-4683-823e-05fce4f6f2a0 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:06.433182 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e1561-cba2-4ddc-b6ba-adfba0832f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:06.434451 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e1561-cba2-4ddc-b6ba-adfba0832f2a None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:06.435069 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e1561-cba2-4ddc-b6ba-adfba0832f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:06.435701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e1561-cba2-4ddc-b6ba-adfba0832f2a None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:06.437190 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e1561-cba2-4ddc-b6ba-adfba0832f2a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:06.440230 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e1561-cba2-4ddc-b6ba-adfba0832f2a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:06.455522 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b955fb-8c2f-43d5-ba16-ce7f29b64c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:06.456171 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b955fb-8c2f-43d5-ba16-ce7f29b64c3d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:06.456467 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b955fb-8c2f-43d5-ba16-ce7f29b64c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:06.456741 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b955fb-8c2f-43d5-ba16-ce7f29b64c3d None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:06.457542 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b955fb-8c2f-43d5-ba16-ce7f29b64c3d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:06.457824 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b955fb-8c2f-43d5-ba16-ce7f29b64c3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:06.459137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b955fb-8c2f-43d5-ba16-ce7f29b64c3d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:06.469570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 117/241] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 13:53:06 2022] POST /identity/v3/users => generated 293 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:53:06.482831 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 125/242] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 13:53:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:06.490553 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e1561-cba2-4ddc-b6ba-adfba0832f2a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:06.625541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e1561-cba2-4ddc-b6ba-adfba0832f2a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:06.659950 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:06.665506 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cdf03a-3937-446e-b11b-e6510850de44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:06.666620 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdf03a-3937-446e-b11b-e6510850de44 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:06.666903 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdf03a-3937-446e-b11b-e6510850de44 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:06.667183 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdf03a-3937-446e-b11b-e6510850de44 None admin] PATH_INFO: `/v3/users/ed7c48945e7a45a68a0e4156685ed750` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:06.685405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cdf03a-3937-446e-b11b-e6510850de44 None admin] RBAC: Authorizing `identity:get_user(user_id=ed7c48945e7a45a68a0e4156685ed750)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:06.687154 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cdf03a-3937-446e-b11b-e6510850de44 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:06.689352 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 118/243] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:53:06 2022] GET /identity/v3/users/ed7c48945e7a45a68a0e4156685ed750 => generated 297 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:53:06.697809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b955fb-8c2f-43d5-ba16-ce7f29b64c3d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:06.869159 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:06.884615 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b955fb-8c2f-43d5-ba16-ce7f29b64c3d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:06.937204 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121bdb7e-fb23-4fef-881f-54e1a1aacfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:06.938199 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121bdb7e-fb23-4fef-881f-54e1a1aacfe3 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:06.938614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121bdb7e-fb23-4fef-881f-54e1a1aacfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:06.938987 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121bdb7e-fb23-4fef-881f-54e1a1aacfe3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:07.020433 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121bdb7e-fb23-4fef-881f-54e1a1aacfe3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:07.023053 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121bdb7e-fb23-4fef-881f-54e1a1aacfe3 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:07.048899 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c706bd8-590b-4b4a-99ee-59b2988bfd3e None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:07.049459 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c706bd8-590b-4b4a-99ee-59b2988bfd3e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:07.049728 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c706bd8-590b-4b4a-99ee-59b2988bfd3e None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:07.052177 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 119/244] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:07.063526 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb1998-d536-442f-b9df-8cdb26f6252d None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:07.064117 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb1998-d536-442f-b9df-8cdb26f6252d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:07.064627 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb1998-d536-442f-b9df-8cdb26f6252d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:07.103894 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-cfcb1998-d536-442f-b9df-8cdb26f6252d None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:07.106111 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-cfcb1998-d536-442f-b9df-8cdb26f6252d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:07.160792 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:07.173572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 126/245] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:53:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 13:53:07.180501 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121bdb7e-fb23-4fef-881f-54e1a1aacfe3 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:07.229432 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 120/246] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:07.257963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb1998-d536-442f-b9df-8cdb26f6252d None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:07.314934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121bdb7e-fb23-4fef-881f-54e1a1aacfe3 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:07.354877 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b0296-6aee-4afd-b241-1307d6962f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:07.355812 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b0296-6aee-4afd-b241-1307d6962f61 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:07.356249 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b0296-6aee-4afd-b241-1307d6962f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:07.356676 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b0296-6aee-4afd-b241-1307d6962f61 None admin] PATH_INFO: `/v3/projects/7489ab03e4a8475cab57be63c14ec0c0` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:07.358128 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5b0296-6aee-4afd-b241-1307d6962f61 None admin] RBAC: Authorizing `identity:get_project(project_id=7489ab03e4a8475cab57be63c14ec0c0)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:07.360153 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5b0296-6aee-4afd-b241-1307d6962f61 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:07.362294 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 127/247] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:53:07 2022] GET /identity/v3/projects/7489ab03e4a8475cab57be63c14ec0c0 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:07.371572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b0296-6aee-4afd-b241-1307d6962f61 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:07.417005 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb1998-d536-442f-b9df-8cdb26f6252d None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:07.451900 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:07.471569 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d15bc-b009-41c4-b2c0-bee776badf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:07.472426 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d15bc-b009-41c4-b2c0-bee776badf9d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:07.472808 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d15bc-b009-41c4-b2c0-bee776badf9d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:07.473179 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d15bc-b009-41c4-b2c0-bee776badf9d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:07.483421 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:07.502049 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063d15bc-b009-41c4-b2c0-bee776badf9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:07.507452 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-063d15bc-b009-41c4-b2c0-bee776badf9d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:53:07.507452 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:07.507452 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:07.507452 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:07.507452 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:07.507452 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:07.507452 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:07.507452 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:07.507452 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:07.507452 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:07.507452 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:07.507452 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:07.507452 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:07.507452 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:07.507452 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:07.507452 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:07.508582 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:07.508582 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:07.508582 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:07.508582 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:07.508582 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:07.508582 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:07.508582 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:07.508582 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:07.508582 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:07.508582 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:07.508582 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:07.508582 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:07.508582 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:07.508582 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:07.508582 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:07.508582 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:07.509610 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:07.509610 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:07.509610 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:07.509610 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:07.509610 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:07.509610 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:07.509610 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:07.509610 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:07.509610 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:07.509610 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:07.509610 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:53:07.509610 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:53:07.514057 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 121/248] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:53:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:07.520511 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d15bc-b009-41c4-b2c0-bee776badf9d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:07.536706 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b0296-6aee-4afd-b241-1307d6962f61 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:07.583009 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ecd52c-be0d-4899-a2a6-3c0fb32318f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:07.583822 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ecd52c-be0d-4899-a2a6-3c0fb32318f4 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:07.584119 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ecd52c-be0d-4899-a2a6-3c0fb32318f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:07.584455 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ecd52c-be0d-4899-a2a6-3c0fb32318f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:07.585268 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ecd52c-be0d-4899-a2a6-3c0fb32318f4 None admin] RBAC: Adding query filter params (role.id=9db7d52dd9514261a5a52591fbf1feef, scope.project.id=7489ab03e4a8475cab57be63c14ec0c0, user.id=ed7c48945e7a45a68a0e4156685ed750) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:07.585586 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ecd52c-be0d-4899-a2a6-3c0fb32318f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:07.587544 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ecd52c-be0d-4899-a2a6-3c0fb32318f4 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:07.596708 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 128/249] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:53:07 2022] GET /identity/v3/role_assignments?user.id=ed7c48945e7a45a68a0e4156685ed750&scope.project.id=7489ab03e4a8475cab57be63c14ec0c0&role.id=9db7d52dd9514261a5a52591fbf1feef => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:53:07.739102 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de7e58-0ead-44a9-ad34-3432f5d31b46 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:07.739552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de7e58-0ead-44a9-ad34-3432f5d31b46 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:07.740534 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de7e58-0ead-44a9-ad34-3432f5d31b46 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:07.742280 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d15bc-b009-41c4-b2c0-bee776badf9d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:07.745506 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 129/250] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:07.753196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ebe2e-dacf-4fc9-bf2e-b41b427e0229 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:07.753503 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ebe2e-dacf-4fc9-bf2e-b41b427e0229 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:07.754543 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ebe2e-dacf-4fc9-bf2e-b41b427e0229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:07.801569 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-6a6ebe2e-dacf-4fc9-bf2e-b41b427e0229 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:07.803293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-6a6ebe2e-dacf-4fc9-bf2e-b41b427e0229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:07.831837 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3c7c4-68cb-410d-bb65-bb6f6c6c8d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:07.832570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3c7c4-68cb-410d-bb65-bb6f6c6c8d90 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:07.832865 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3c7c4-68cb-410d-bb65-bb6f6c6c8d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:07.833206 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3c7c4-68cb-410d-bb65-bb6f6c6c8d90 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:07.834204 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3c7c4-68cb-410d-bb65-bb6f6c6c8d90 None admin] RBAC: Adding query filter params (name=member) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:07.834525 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3c7c4-68cb-410d-bb65-bb6f6c6c8d90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:07.836196 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3c7c4-68cb-410d-bb65-bb6f6c6c8d90 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:07.856473 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 122/251] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 13:53:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:07.862122 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3c7c4-68cb-410d-bb65-bb6f6c6c8d90 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:07.946510 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 130/252] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:07.955377 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:08.001443 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ebe2e-dacf-4fc9-bf2e-b41b427e0229 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:08.058176 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3c7c4-68cb-410d-bb65-bb6f6c6c8d90 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:08.113057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:08.152525 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2370e289-4560-4db4-9a93-2ee9e906d607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:08.153190 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370e289-4560-4db4-9a93-2ee9e906d607 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:08.153468 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370e289-4560-4db4-9a93-2ee9e906d607 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:08.153744 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370e289-4560-4db4-9a93-2ee9e906d607 None admin] PATH_INFO: `/v3/users/7822533aa26c4490a849a4da2666c8f5` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:08.154958 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2370e289-4560-4db4-9a93-2ee9e906d607 None admin] RBAC: Authorizing `identity:get_user(user_id=7822533aa26c4490a849a4da2666c8f5)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:08.164710 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2370e289-4560-4db4-9a93-2ee9e906d607 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:08.166927 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 123/253] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:53:07 2022] GET /identity/v3/users/7822533aa26c4490a849a4da2666c8f5 => generated 297 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:53:08.177066 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ebe2e-dacf-4fc9-bf2e-b41b427e0229 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:08.177604 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2370e289-4560-4db4-9a93-2ee9e906d607 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:08.213512 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49de138-e62f-4e82-a414-c4f83dd2def2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:08.214173 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49de138-e62f-4e82-a414-c4f83dd2def2 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:08.214464 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49de138-e62f-4e82-a414-c4f83dd2def2 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:08.214722 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49de138-e62f-4e82-a414-c4f83dd2def2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:08.226029 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49de138-e62f-4e82-a414-c4f83dd2def2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:08.231320 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-a49de138-e62f-4e82-a414-c4f83dd2def2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:53:08.231320 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:08.231320 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:08.231320 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:08.231320 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:08.231320 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:08.231320 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:08.231320 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:08.231320 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:08.231320 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:08.231320 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:08.231320 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:08.231320 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:08.231320 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:08.231320 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:08.231320 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:08.232511 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:08.232511 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:08.232511 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:08.232511 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:08.232511 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:08.232511 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:08.232511 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:08.232511 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:08.232511 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:08.232511 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:08.232511 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:08.232511 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:08.232511 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:08.232511 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:08.232511 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:08.232511 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:08.233326 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:08.233326 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:08.233326 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:08.233326 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:08.233326 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:08.233326 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:08.233326 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:08.233326 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:08.233326 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:08.233326 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:08.233326 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:53:08.233326 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:08.233326 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 131/254] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:53:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:08.238645 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49de138-e62f-4e82-a414-c4f83dd2def2 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:08.348553 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49de138-e62f-4e82-a414-c4f83dd2def2 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:08.363450 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:08.385895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eee7e6-1070-4774-a5f4-0b3f4cbc3a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:08.386531 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eee7e6-1070-4774-a5f4-0b3f4cbc3a72 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:08.386812 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eee7e6-1070-4774-a5f4-0b3f4cbc3a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:08.387076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eee7e6-1070-4774-a5f4-0b3f4cbc3a72 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:08.388204 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eee7e6-1070-4774-a5f4-0b3f4cbc3a72 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:08.388541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eee7e6-1070-4774-a5f4-0b3f4cbc3a72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:08.389992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eee7e6-1070-4774-a5f4-0b3f4cbc3a72 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:08.399698 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 132/255] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:53:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:08.406075 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eee7e6-1070-4774-a5f4-0b3f4cbc3a72 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:08.454440 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2370e289-4560-4db4-9a93-2ee9e906d607 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:08.486610 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:08.529961 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eee7e6-1070-4774-a5f4-0b3f4cbc3a72 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:08.571371 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd80a7a-8134-47e3-8d7e-0befc030a798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:08.572033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd80a7a-8134-47e3-8d7e-0befc030a798 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:08.572338 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd80a7a-8134-47e3-8d7e-0befc030a798 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:08.572613 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd80a7a-8134-47e3-8d7e-0befc030a798 None admin] PATH_INFO: `/v3/users/8dcd8be2726a4f3eaee54e70a9f00394` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:08.574105 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd80a7a-8134-47e3-8d7e-0befc030a798 None admin] RBAC: Authorizing `identity:get_user(user_id=8dcd8be2726a4f3eaee54e70a9f00394)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:08.575566 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd80a7a-8134-47e3-8d7e-0befc030a798 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:08.577439 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 133/256] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:53:08 2022] GET /identity/v3/users/8dcd8be2726a4f3eaee54e70a9f00394 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:53:08.581091 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da5b54-e03b-4c40-a5bd-f1688e35999d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:08.581761 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da5b54-e03b-4c40-a5bd-f1688e35999d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:08.582119 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da5b54-e03b-4c40-a5bd-f1688e35999d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:08.582399 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da5b54-e03b-4c40-a5bd-f1688e35999d None admin] PATH_INFO: `/v3/projects/7489ab03e4a8475cab57be63c14ec0c0` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:08.583521 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd80a7a-8134-47e3-8d7e-0befc030a798 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:08.584999 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9da5b54-e03b-4c40-a5bd-f1688e35999d None admin] RBAC: Authorizing `identity:get_project(project_id=7489ab03e4a8475cab57be63c14ec0c0)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:08.586783 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9da5b54-e03b-4c40-a5bd-f1688e35999d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:08.588630 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 124/257] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:53:08 2022] GET /identity/v3/projects/7489ab03e4a8475cab57be63c14ec0c0 => generated 302 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:08.595055 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da5b54-e03b-4c40-a5bd-f1688e35999d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:08.638493 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:08.703013 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd80a7a-8134-47e3-8d7e-0befc030a798 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:08.743525 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc256e-9e55-4003-80d7-1372724b8d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:08.744637 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc256e-9e55-4003-80d7-1372724b8d52 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:08.745109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc256e-9e55-4003-80d7-1372724b8d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:08.745553 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc256e-9e55-4003-80d7-1372724b8d52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:08.805386 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da5b54-e03b-4c40-a5bd-f1688e35999d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:08.826010 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdc256e-9e55-4003-80d7-1372724b8d52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:08.828530 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdc256e-9e55-4003-80d7-1372724b8d52 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:08.873027 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7b0aa-be19-4a4e-a2c8-0d81fff1f9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:08.873819 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7b0aa-be19-4a4e-a2c8-0d81fff1f9eb None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:08.874171 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7b0aa-be19-4a4e-a2c8-0d81fff1f9eb None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:08.874590 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7b0aa-be19-4a4e-a2c8-0d81fff1f9eb None admin] PATH_INFO: `/v3/projects/7489ab03e4a8475cab57be63c14ec0c0/users/7822533aa26c4490a849a4da2666c8f5/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:08.878293 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d7b0aa-be19-4a4e-a2c8-0d81fff1f9eb None admin] RBAC: Authorizing `identity:create_grant(project_id=7489ab03e4a8475cab57be63c14ec0c0, user_id=7822533aa26c4490a849a4da2666c8f5, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:08.879576 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:08.881229 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d7b0aa-be19-4a4e-a2c8-0d81fff1f9eb None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:08.910900 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 125/258] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 13:53:08 2022] PUT /identity/v3/projects/7489ab03e4a8475cab57be63c14ec0c0/users/7822533aa26c4490a849a4da2666c8f5/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 13:53:08.912183 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639537eb-b5dc-404a-846a-d91a793258a3 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:08.912501 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639537eb-b5dc-404a-846a-d91a793258a3 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:08.912781 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639537eb-b5dc-404a-846a-d91a793258a3 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:08.924696 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 126/259] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:08 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:08.926159 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d1b0c-fd0f-4d03-b72e-239d1c3bb80b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:08.926447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d1b0c-fd0f-4d03-b72e-239d1c3bb80b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:08.926729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d1b0c-fd0f-4d03-b72e-239d1c3bb80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:08.947711 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 134/260] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:53:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:53:08.955039 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc256e-9e55-4003-80d7-1372724b8d52 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:08.973386 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b82d1b0c-fd0f-4d03-b72e-239d1c3bb80b None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:08.974627 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b82d1b0c-fd0f-4d03-b72e-239d1c3bb80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:09.043404 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:09.076180 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc256e-9e55-4003-80d7-1372724b8d52 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:09.086985 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 127/261] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:09.111355 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d1b0c-fd0f-4d03-b72e-239d1c3bb80b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:09.120205 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5caae78-d997-48e7-803d-11c322daa7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:09.120914 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5caae78-d997-48e7-803d-11c322daa7e4 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:09.121928 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5caae78-d997-48e7-803d-11c322daa7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:09.122203 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5caae78-d997-48e7-803d-11c322daa7e4 None admin] PATH_INFO: `/v3/projects/7489ab03e4a8475cab57be63c14ec0c0` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:09.123474 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5caae78-d997-48e7-803d-11c322daa7e4 None admin] RBAC: Authorizing `identity:get_project(project_id=7489ab03e4a8475cab57be63c14ec0c0)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:09.125872 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5caae78-d997-48e7-803d-11c322daa7e4 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:09.128298 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 135/262] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:53:08 2022] GET /identity/v3/projects/7489ab03e4a8475cab57be63c14ec0c0 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:09.134651 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5caae78-d997-48e7-803d-11c322daa7e4 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:09.164380 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:09.302305 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d1b0c-fd0f-4d03-b72e-239d1c3bb80b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:09.337117 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5caae78-d997-48e7-803d-11c322daa7e4 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:09.344422 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c7a95-6f0f-4ffa-b17f-3951e536144c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:09.345078 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c7a95-6f0f-4ffa-b17f-3951e536144c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:09.345356 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c7a95-6f0f-4ffa-b17f-3951e536144c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:09.345623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c7a95-6f0f-4ffa-b17f-3951e536144c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:09.357772 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3c7a95-6f0f-4ffa-b17f-3951e536144c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:09.362829 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-ec3c7a95-6f0f-4ffa-b17f-3951e536144c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:53:09.362829 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:09.362829 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:09.362829 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:09.362829 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:09.362829 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:09.362829 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:09.362829 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:09.362829 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:09.362829 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:09.362829 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:09.362829 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:09.362829 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:09.362829 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:09.362829 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:09.362829 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:09.363763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:09.363763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:09.363763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:09.363763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:09.363763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:09.363763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:09.363763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:09.363763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:09.363763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:09.363763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:09.363763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:09.363763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:09.363763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:09.363763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:09.363763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:09.363763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:09.364648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:09.364648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:09.364648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:09.364648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:09.364648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:09.364648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:09.364648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:09.364648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:09.364648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:09.364648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:09.364648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:53:09.364648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:53:09.364648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 128/263] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 13:53:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:09.375372 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c7a95-6f0f-4ffa-b17f-3951e536144c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:09.386217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b5b55-0b4f-4d82-b3f4-e6b9fea643b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:09.386854 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b5b55-0b4f-4d82-b3f4-e6b9fea643b6 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:09.387138 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b5b55-0b4f-4d82-b3f4-e6b9fea643b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:09.387411 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b5b55-0b4f-4d82-b3f4-e6b9fea643b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:09.388217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91b5b55-0b4f-4d82-b3f4-e6b9fea643b6 None admin] RBAC: Adding query filter params (role.id=3e1a9ff016c840c5b51ef7e3da9900ac, scope.project.id=7489ab03e4a8475cab57be63c14ec0c0, user.id=8dcd8be2726a4f3eaee54e70a9f00394) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:09.388567 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91b5b55-0b4f-4d82-b3f4-e6b9fea643b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:09.391109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91b5b55-0b4f-4d82-b3f4-e6b9fea643b6 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:09.404839 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 136/264] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:53:09 2022] GET /identity/v3/role_assignments?user.id=8dcd8be2726a4f3eaee54e70a9f00394&scope.project.id=7489ab03e4a8475cab57be63c14ec0c0&role.id=3e1a9ff016c840c5b51ef7e3da9900ac => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:09.449788 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:09.584231 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c7a95-6f0f-4ffa-b17f-3951e536144c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:09.622193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff0429f-50ef-4e39-aed8-5ce074879ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:09.622819 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0429f-50ef-4e39-aed8-5ce074879ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:09.623098 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0429f-50ef-4e39-aed8-5ce074879ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:09.623424 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0429f-50ef-4e39-aed8-5ce074879ea9 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:09.624142 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff0429f-50ef-4e39-aed8-5ce074879ea9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:09.624695 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff0429f-50ef-4e39-aed8-5ce074879ea9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:09.626004 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff0429f-50ef-4e39-aed8-5ce074879ea9 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:09.638648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 129/265] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:53:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:09.641821 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b5b55-0b4f-4d82-b3f4-e6b9fea643b6 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:09.732768 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:09.773437 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b5b55-0b4f-4d82-b3f4-e6b9fea643b6 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:09.818786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dee946-925c-4e2a-b603-6e9c84787f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:09.819510 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dee946-925c-4e2a-b603-6e9c84787f1f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:09.819789 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dee946-925c-4e2a-b603-6e9c84787f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:09.820057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dee946-925c-4e2a-b603-6e9c84787f1f None admin] PATH_INFO: `/v3/users/8dcd8be2726a4f3eaee54e70a9f00394` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:09.821491 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dee946-925c-4e2a-b603-6e9c84787f1f None admin] RBAC: Authorizing `identity:get_user(user_id=8dcd8be2726a4f3eaee54e70a9f00394)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:09.822965 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dee946-925c-4e2a-b603-6e9c84787f1f None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:09.824985 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 137/266] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:53:09 2022] GET /identity/v3/users/8dcd8be2726a4f3eaee54e70a9f00394 => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:09.831451 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff0429f-50ef-4e39-aed8-5ce074879ea9 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:09.960659 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:09.971736 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff0429f-50ef-4e39-aed8-5ce074879ea9 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:10.029012 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5ef3bb-35a5-4920-aba5-00d094d08557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:10.029540 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ef3bb-35a5-4920-aba5-00d094d08557 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:10.029820 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ef3bb-35a5-4920-aba5-00d094d08557 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:10.030091 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ef3bb-35a5-4920-aba5-00d094d08557 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:10.109722 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5ef3bb-35a5-4920-aba5-00d094d08557 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:10.111213 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5ef3bb-35a5-4920-aba5-00d094d08557 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:10.232709 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:10.235013 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 130/267] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:53:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:53:10.236679 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dee946-925c-4e2a-b603-6e9c84787f1f None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:10.377486 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dee946-925c-4e2a-b603-6e9c84787f1f None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:10.425674 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ef9f6-5593-4aa3-8c0a-59b804a7b9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:10.426424 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ef9f6-5593-4aa3-8c0a-59b804a7b9bb None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:10.426705 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ef9f6-5593-4aa3-8c0a-59b804a7b9bb None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:10.426998 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ef9f6-5593-4aa3-8c0a-59b804a7b9bb None admin] PATH_INFO: `/v3/projects/7489ab03e4a8475cab57be63c14ec0c0` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:10.428428 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160ef9f6-5593-4aa3-8c0a-59b804a7b9bb None admin] RBAC: Authorizing `identity:get_project(project_id=7489ab03e4a8475cab57be63c14ec0c0)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:10.430066 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160ef9f6-5593-4aa3-8c0a-59b804a7b9bb None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:10.431958 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 138/268] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:53:10 2022] GET /identity/v3/projects/7489ab03e4a8475cab57be63c14ec0c0 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:10.438405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5ef3bb-35a5-4920-aba5-00d094d08557 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:10.571543 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5ef3bb-35a5-4920-aba5-00d094d08557 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:10.614340 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02754a36-05ee-4551-90e2-3a55ab47ac9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:10.614991 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02754a36-05ee-4551-90e2-3a55ab47ac9b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:10.615269 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02754a36-05ee-4551-90e2-3a55ab47ac9b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:10.615543 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02754a36-05ee-4551-90e2-3a55ab47ac9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:10.616466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02754a36-05ee-4551-90e2-3a55ab47ac9b None admin] RBAC: Adding query filter params (role.id=60efb70fa7354ebfb04aa5e9bdad2a78, scope.project.id=7489ab03e4a8475cab57be63c14ec0c0, user.id=8dcd8be2726a4f3eaee54e70a9f00394) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:10.616772 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02754a36-05ee-4551-90e2-3a55ab47ac9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:10.618283 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02754a36-05ee-4551-90e2-3a55ab47ac9b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:10.625757 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 131/269] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:53:10 2022] GET /identity/v3/role_assignments?user.id=8dcd8be2726a4f3eaee54e70a9f00394&scope.project.id=7489ab03e4a8475cab57be63c14ec0c0&role.id=60efb70fa7354ebfb04aa5e9bdad2a78 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:10.637936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c280a-5073-42eb-9829-c0b72d35baaf None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:10.638154 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c280a-5073-42eb-9829-c0b72d35baaf None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:10.638446 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c280a-5073-42eb-9829-c0b72d35baaf None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:10.639854 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 139/270] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:10.646362 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263e1fe-9523-4909-85a0-fcfd96dace38 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:10.646739 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263e1fe-9523-4909-85a0-fcfd96dace38 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:10.646962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263e1fe-9523-4909-85a0-fcfd96dace38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:10.655286 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548764d-6d89-48b7-828b-29b257834826 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:10.655760 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548764d-6d89-48b7-828b-29b257834826 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:10.655955 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548764d-6d89-48b7-828b-29b257834826 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:10.657623 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 140/271] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:10.666151 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626b735-f3af-4f55-bf98-b39b7bb3eee2 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:10.666429 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626b735-f3af-4f55-bf98-b39b7bb3eee2 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:10.666703 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626b735-f3af-4f55-bf98-b39b7bb3eee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:10.677994 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c263e1fe-9523-4909-85a0-fcfd96dace38 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:10.679913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c263e1fe-9523-4909-85a0-fcfd96dace38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:10.709507 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-7626b735-f3af-4f55-bf98-b39b7bb3eee2 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:10.710915 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7626b735-f3af-4f55-bf98-b39b7bb3eee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:10.724789 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:10.770198 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:10.815884 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 132/272] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:10.831235 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 141/273] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:10.852706 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c263e1fe-9523-4909-85a0-fcfd96dace38 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:10.857142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7626b735-f3af-4f55-bf98-b39b7bb3eee2 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:10.976891 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7626b735-f3af-4f55-bf98-b39b7bb3eee2 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:11.019248 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68495ba8-fd09-4ea5-bef7-443ab65728fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:11.020226 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68495ba8-fd09-4ea5-bef7-443ab65728fe None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:11.020672 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68495ba8-fd09-4ea5-bef7-443ab65728fe None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:11.021155 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68495ba8-fd09-4ea5-bef7-443ab65728fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:11.023699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68495ba8-fd09-4ea5-bef7-443ab65728fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:11.025458 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68495ba8-fd09-4ea5-bef7-443ab65728fe None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:11.033050 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 142/274] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:53:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:11.033050 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da53a8d-bfb7-4204-8f6d-88018e15ddf1 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:11.033050 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da53a8d-bfb7-4204-8f6d-88018e15ddf1 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:11.033050 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da53a8d-bfb7-4204-8f6d-88018e15ddf1 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:11.033050 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 143/275] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:11.034609 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da53a8d-bfb7-4204-8f6d-88018e15ddf1 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:11.050586 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:11.085637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c263e1fe-9523-4909-85a0-fcfd96dace38 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:11.135466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff434de0-46b0-430f-a752-f77bf9eff0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:11.136505 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff434de0-46b0-430f-a752-f77bf9eff0db None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:11.136505 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff434de0-46b0-430f-a752-f77bf9eff0db None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:11.136756 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff434de0-46b0-430f-a752-f77bf9eff0db None admin] PATH_INFO: `/v3/roles/member` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:11.150663 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff434de0-46b0-430f-a752-f77bf9eff0db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:11.156324 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-ff434de0-46b0-430f-a752-f77bf9eff0db None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:53:11.156324 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:11.156324 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:11.156324 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:11.156324 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:11.156324 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:11.156324 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:11.156324 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:11.156324 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:11.156324 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:11.156324 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:11.156324 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:11.156324 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:11.156324 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:11.156324 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:11.156324 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:11.157325 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:11.157325 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:11.157325 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:11.157325 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:11.157325 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:11.157325 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:11.157325 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:11.157325 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:11.157325 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:11.157325 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:11.157325 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:11.157325 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:11.157325 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:11.157325 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:11.157325 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:11.157325 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:11.158149 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:11.158149 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:11.158149 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:11.158149 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:11.158149 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:11.158149 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:11.158149 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:11.158149 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:11.158149 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:11.158149 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:11.158149 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:53:11.158149 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:53:11.158149 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 133/276] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:53:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:11.158836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea832da-0212-4777-9974-d954940b440b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:11.159703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea832da-0212-4777-9974-d954940b440b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:11.159703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea832da-0212-4777-9974-d954940b440b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:11.179107 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8ea832da-0212-4777-9974-d954940b440b None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:11.180648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8ea832da-0212-4777-9974-d954940b440b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:11.194746 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da53a8d-bfb7-4204-8f6d-88018e15ddf1 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:11.228331 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:11.231291 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f421d-c2f0-457d-b1a4-2f6b2c51f426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:11.231930 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f421d-c2f0-457d-b1a4-2f6b2c51f426 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:11.232345 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f421d-c2f0-457d-b1a4-2f6b2c51f426 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:11.232628 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f421d-c2f0-457d-b1a4-2f6b2c51f426 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:11.233356 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f421d-c2f0-457d-b1a4-2f6b2c51f426 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:11.234937 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f421d-c2f0-457d-b1a4-2f6b2c51f426 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:11.264810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 144/277] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 13:53:11 2022] POST /identity/v3/users => generated 293 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 13:53:11.264810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f421d-c2f0-457d-b1a4-2f6b2c51f426 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:11.277690 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 134/278] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:11.301955 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea832da-0212-4777-9974-d954940b440b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:11.344294 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:11.391962 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f421d-c2f0-457d-b1a4-2f6b2c51f426 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:11.416498 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea832da-0212-4777-9974-d954940b440b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:11.432321 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357a58d-71b8-4e92-a3de-dd96f5933c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:11.433077 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357a58d-71b8-4e92-a3de-dd96f5933c44 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:11.433474 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357a58d-71b8-4e92-a3de-dd96f5933c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:11.433748 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357a58d-71b8-4e92-a3de-dd96f5933c44 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:11.434610 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357a58d-71b8-4e92-a3de-dd96f5933c44 None admin] RBAC: Adding query filter params (name=member) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:11.434923 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357a58d-71b8-4e92-a3de-dd96f5933c44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:11.436372 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357a58d-71b8-4e92-a3de-dd96f5933c44 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:11.446841 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 145/279] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 13:53:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:11.453518 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357a58d-71b8-4e92-a3de-dd96f5933c44 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:11.459852 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e48a72-85e5-4cd8-bd61-d0b34d6f4f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:11.460072 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e48a72-85e5-4cd8-bd61-d0b34d6f4f62 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:11.460290 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e48a72-85e5-4cd8-bd61-d0b34d6f4f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:11.460666 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e48a72-85e5-4cd8-bd61-d0b34d6f4f62 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:11.474345 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e48a72-85e5-4cd8-bd61-d0b34d6f4f62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:11.479447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-e2e48a72-85e5-4cd8-bd61-d0b34d6f4f62 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:53:11.479447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:11.479447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:11.479447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:11.479447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:11.479447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:11.479447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:11.479447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:11.479447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:11.479447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:11.479447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:11.479447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:11.479447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:11.479447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:11.479447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:11.479447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:11.480747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:11.480747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:11.480747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:11.480747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:11.480747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:11.480747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:11.480747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:11.480747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:11.480747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:11.480747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:11.480747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:11.480747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:11.480747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:11.480747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:11.480747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:11.480747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:11.482045 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:11.482045 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:11.482045 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:11.482045 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:11.482045 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:11.482045 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:11.482045 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:11.482045 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:11.482045 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:11.482045 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:11.482045 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:53:11.482045 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:53:11.482045 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 135/280] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:53:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:11.487644 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e48a72-85e5-4cd8-bd61-d0b34d6f4f62 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:11.539182 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:11.612524 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357a58d-71b8-4e92-a3de-dd96f5933c44 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:11.637236 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e48a72-85e5-4cd8-bd61-d0b34d6f4f62 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:11.652291 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:11.669402 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65181c-f931-4930-9506-2961106449f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:11.670141 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65181c-f931-4930-9506-2961106449f9 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:11.670423 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65181c-f931-4930-9506-2961106449f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:11.670693 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65181c-f931-4930-9506-2961106449f9 None admin] PATH_INFO: `/v3/users/d6d0746eb63c46ffbcc1ec5ed80e20e5` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:11.691136 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db663b-e346-4ed3-a8b8-3e31314fa166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:11.691778 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db663b-e346-4ed3-a8b8-3e31314fa166 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:11.692052 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db663b-e346-4ed3-a8b8-3e31314fa166 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:11.692231 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf65181c-f931-4930-9506-2961106449f9 None admin] RBAC: Authorizing `identity:get_user(user_id=d6d0746eb63c46ffbcc1ec5ed80e20e5)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:11.692358 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db663b-e346-4ed3-a8b8-3e31314fa166 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:11.693075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40db663b-e346-4ed3-a8b8-3e31314fa166 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:11.693370 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40db663b-e346-4ed3-a8b8-3e31314fa166 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:11.693782 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf65181c-f931-4930-9506-2961106449f9 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:11.694746 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40db663b-e346-4ed3-a8b8-3e31314fa166 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:11.695731 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 146/281] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:53:11 2022] GET /identity/v3/users/d6d0746eb63c46ffbcc1ec5ed80e20e5 => generated 293 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:11.701929 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65181c-f931-4930-9506-2961106449f9 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:11.706216 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 136/282] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 13:53:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:11.713640 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db663b-e346-4ed3-a8b8-3e31314fa166 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:11.844249 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65181c-f931-4930-9506-2961106449f9 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:11.851648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:11.857932 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db663b-e346-4ed3-a8b8-3e31314fa166 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:11.891657 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dfd739-dc6a-43a0-8f86-3e251f6a903c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:11.893130 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfd739-dc6a-43a0-8f86-3e251f6a903c None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:11.894346 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfd739-dc6a-43a0-8f86-3e251f6a903c None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:11.894977 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfd739-dc6a-43a0-8f86-3e251f6a903c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:11.897003 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dfd739-dc6a-43a0-8f86-3e251f6a903c None admin] RBAC: Adding query filter params (role.id=75aff8cb4cf748a2adbd6c3f141fe8ac, user.id=d6d0746eb63c46ffbcc1ec5ed80e20e5, scope.system=all) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:11.898282 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dfd739-dc6a-43a0-8f86-3e251f6a903c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:11.900841 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dfd739-dc6a-43a0-8f86-3e251f6a903c None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:11.908319 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 147/283] 10.222.0.45 () {56 vars in 1308 bytes} [Mon Aug 1 13:53:11 2022] GET /identity/v3/role_assignments?user.id=d6d0746eb63c46ffbcc1ec5ed80e20e5&scope.system=all&role.id=75aff8cb4cf748a2adbd6c3f141fe8ac => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:11.909948 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b074fb-179b-4a9f-bf88-d8d4cc673a01 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:11.910441 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b074fb-179b-4a9f-bf88-d8d4cc673a01 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:11.910915 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b074fb-179b-4a9f-bf88-d8d4cc673a01 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:11.912644 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 148/284] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:11.920355 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742d7c2-6526-4571-b568-c9e9148de339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:11.920355 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742d7c2-6526-4571-b568-c9e9148de339 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:11.920355 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742d7c2-6526-4571-b568-c9e9148de339 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:11.920355 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742d7c2-6526-4571-b568-c9e9148de339 None admin] PATH_INFO: `/v3/users/ed7c48945e7a45a68a0e4156685ed750` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:11.920836 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9523e-d3c5-4eab-8fe2-1397f7e6e947 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:11.920836 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9523e-d3c5-4eab-8fe2-1397f7e6e947 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:11.920980 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742d7c2-6526-4571-b568-c9e9148de339 None admin] RBAC: Authorizing `identity:get_user(user_id=ed7c48945e7a45a68a0e4156685ed750)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:11.921053 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9523e-d3c5-4eab-8fe2-1397f7e6e947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:11.923687 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742d7c2-6526-4571-b568-c9e9148de339 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:11.924364 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 137/285] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:53:11 2022] GET /identity/v3/users/ed7c48945e7a45a68a0e4156685ed750 => generated 297 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:11.932346 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742d7c2-6526-4571-b568-c9e9148de339 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:11.959720 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-57b9523e-d3c5-4eab-8fe2-1397f7e6e947 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:11.965384 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-57b9523e-d3c5-4eab-8fe2-1397f7e6e947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:12.058699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:12.082109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 149/286] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:12.106242 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9523e-d3c5-4eab-8fe2-1397f7e6e947 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:12.177205 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742d7c2-6526-4571-b568-c9e9148de339 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:12.225325 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe37fc-3ea7-455e-8b1c-558d62e68c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:12.226233 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe37fc-3ea7-455e-8b1c-558d62e68c1b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:12.226341 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe37fc-3ea7-455e-8b1c-558d62e68c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:12.226543 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe37fc-3ea7-455e-8b1c-558d62e68c1b None admin] PATH_INFO: `/v3/projects/7489ab03e4a8475cab57be63c14ec0c0` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:12.228994 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fe37fc-3ea7-455e-8b1c-558d62e68c1b None admin] RBAC: Authorizing `identity:get_project(project_id=7489ab03e4a8475cab57be63c14ec0c0)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:12.230664 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fe37fc-3ea7-455e-8b1c-558d62e68c1b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:12.240432 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 138/287] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:53:11 2022] GET /identity/v3/projects/7489ab03e4a8475cab57be63c14ec0c0 => generated 302 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:12.249217 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe37fc-3ea7-455e-8b1c-558d62e68c1b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:12.262234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9523e-d3c5-4eab-8fe2-1397f7e6e947 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:12.303167 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8600381f-3b5c-40c2-a4ba-ce7cee8b33d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:12.305013 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600381f-3b5c-40c2-a4ba-ce7cee8b33d5 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:12.305483 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600381f-3b5c-40c2-a4ba-ce7cee8b33d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:12.305937 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600381f-3b5c-40c2-a4ba-ce7cee8b33d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:12.320646 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8600381f-3b5c-40c2-a4ba-ce7cee8b33d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:12.327293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-8600381f-3b5c-40c2-a4ba-ce7cee8b33d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:53:12.327293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:12.327293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:12.327293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:12.327293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:12.327293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:12.327293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:12.327293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:12.327293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:12.327293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:12.327293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:12.327293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:12.327293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:12.327293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:12.327293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:12.327293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:12.328496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:12.328496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:12.328496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:12.328496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:12.328496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:12.328496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:12.328496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:12.328496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:12.328496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:12.328496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:12.328496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:12.328496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:12.328496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:12.328496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:12.328496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:12.328496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:12.329340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:12.329340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:12.329340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:12.329340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:12.329340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:12.329340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:12.329340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:12.329340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:12.329340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:12.329340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:12.329340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:53:12.329340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:12.332159 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 150/288] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:53:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:53:12.341216 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8600381f-3b5c-40c2-a4ba-ce7cee8b33d5 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:12.343126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:12.412797 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:12.475428 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe37fc-3ea7-455e-8b1c-558d62e68c1b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:12.525174 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8600381f-3b5c-40c2-a4ba-ce7cee8b33d5 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:12.549785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df21102-d82f-414e-a396-58234e568b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:12.550802 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df21102-d82f-414e-a396-58234e568b99 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:12.551284 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df21102-d82f-414e-a396-58234e568b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:12.551748 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df21102-d82f-414e-a396-58234e568b99 None admin] PATH_INFO: `/v3/projects/7489ab03e4a8475cab57be63c14ec0c0/users/ed7c48945e7a45a68a0e4156685ed750/roles/9db7d52dd9514261a5a52591fbf1feef` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:12.554628 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df21102-d82f-414e-a396-58234e568b99 None admin] RBAC: Authorizing `identity:create_grant(project_id=7489ab03e4a8475cab57be63c14ec0c0, user_id=ed7c48945e7a45a68a0e4156685ed750, role_id=9db7d52dd9514261a5a52591fbf1feef)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:12.558018 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df21102-d82f-414e-a396-58234e568b99 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:12.577687 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 139/289] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 13:53:12 2022] PUT /identity/v3/projects/7489ab03e4a8475cab57be63c14ec0c0/users/ed7c48945e7a45a68a0e4156685ed750/roles/9db7d52dd9514261a5a52591fbf1feef => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 13:53:12.616661 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d7b94-c7f7-4495-a75f-5f90c19856c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:12.617313 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d7b94-c7f7-4495-a75f-5f90c19856c5 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:12.617595 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d7b94-c7f7-4495-a75f-5f90c19856c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:12.617865 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d7b94-c7f7-4495-a75f-5f90c19856c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:12.618600 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d7b94-c7f7-4495-a75f-5f90c19856c5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:12.618902 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d7b94-c7f7-4495-a75f-5f90c19856c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:12.620344 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d7b94-c7f7-4495-a75f-5f90c19856c5 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:12.635605 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 151/290] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 13:53:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:53:12.643588 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df21102-d82f-414e-a396-58234e568b99 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:12.764130 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:12.775694 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df21102-d82f-414e-a396-58234e568b99 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:12.819685 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d8e89-aae2-448d-9359-dbc6468e9894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:12.820894 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d8e89-aae2-448d-9359-dbc6468e9894 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:12.821425 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d8e89-aae2-448d-9359-dbc6468e9894 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:12.821986 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d8e89-aae2-448d-9359-dbc6468e9894 None admin] PATH_INFO: `/v3/users/7822533aa26c4490a849a4da2666c8f5` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:12.823477 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4d8e89-aae2-448d-9359-dbc6468e9894 None admin] RBAC: Authorizing `identity:get_user(user_id=7822533aa26c4490a849a4da2666c8f5)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:12.825228 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4d8e89-aae2-448d-9359-dbc6468e9894 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:12.827720 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 140/291] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:53:12 2022] GET /identity/v3/users/7822533aa26c4490a849a4da2666c8f5 => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:12.833661 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d7b94-c7f7-4495-a75f-5f90c19856c5 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:12.990382 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:12.998708 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d7b94-c7f7-4495-a75f-5f90c19856c5 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:13.043871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1833724-14fd-40e5-912c-756082dc2396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:13.044434 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1833724-14fd-40e5-912c-756082dc2396 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:13.044715 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1833724-14fd-40e5-912c-756082dc2396 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:13.044986 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1833724-14fd-40e5-912c-756082dc2396 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:13.139308 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1833724-14fd-40e5-912c-756082dc2396 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:13.141378 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1833724-14fd-40e5-912c-756082dc2396 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:13.359025 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:13.363419 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 152/292] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:53:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:53:13.369467 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d8e89-aae2-448d-9359-dbc6468e9894 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:13.554774 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d8e89-aae2-448d-9359-dbc6468e9894 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:13.612209 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c5e45d-7aac-4e15-8428-a6d69b0b1971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:13.612820 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5e45d-7aac-4e15-8428-a6d69b0b1971 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:13.613101 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5e45d-7aac-4e15-8428-a6d69b0b1971 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:13.613374 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5e45d-7aac-4e15-8428-a6d69b0b1971 None admin] PATH_INFO: `/v3/projects/7489ab03e4a8475cab57be63c14ec0c0` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:13.614669 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c5e45d-7aac-4e15-8428-a6d69b0b1971 None admin] RBAC: Authorizing `identity:get_project(project_id=7489ab03e4a8475cab57be63c14ec0c0)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:13.616579 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c5e45d-7aac-4e15-8428-a6d69b0b1971 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:13.618535 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 141/293] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:53:13 2022] GET /identity/v3/projects/7489ab03e4a8475cab57be63c14ec0c0 => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:13.624690 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1833724-14fd-40e5-912c-756082dc2396 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:13.789087 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1833724-14fd-40e5-912c-756082dc2396 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:13.837992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a8dcc-5944-4f75-8150-0a903a029879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:13.838629 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a8dcc-5944-4f75-8150-0a903a029879 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:13.838910 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a8dcc-5944-4f75-8150-0a903a029879 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:13.839182 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a8dcc-5944-4f75-8150-0a903a029879 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:13.839983 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a8dcc-5944-4f75-8150-0a903a029879 None admin] RBAC: Adding query filter params (role.id=75aff8cb4cf748a2adbd6c3f141fe8ac, scope.project.id=7489ab03e4a8475cab57be63c14ec0c0, user.id=7822533aa26c4490a849a4da2666c8f5) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:13.840307 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a8dcc-5944-4f75-8150-0a903a029879 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:13.841784 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a8dcc-5944-4f75-8150-0a903a029879 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:13.849601 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 153/294] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:53:13 2022] GET /identity/v3/role_assignments?user.id=7822533aa26c4490a849a4da2666c8f5&scope.project.id=7489ab03e4a8475cab57be63c14ec0c0&role.id=75aff8cb4cf748a2adbd6c3f141fe8ac => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:13.960029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104b952-309d-4557-b66a-c01e0211b530 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:13.960446 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104b952-309d-4557-b66a-c01e0211b530 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:13.960815 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104b952-309d-4557-b66a-c01e0211b530 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:13.962682 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 142/295] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:13.968439 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f62895-33b5-443a-b565-6fce55cea7aa None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:13.968822 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f62895-33b5-443a-b565-6fce55cea7aa None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:13.969164 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f62895-33b5-443a-b565-6fce55cea7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:13.989070 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-00f62895-33b5-443a-b565-6fce55cea7aa None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:13.990511 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-00f62895-33b5-443a-b565-6fce55cea7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:14.090763 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:14.100603 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 154/296] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:14.122755 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104b952-309d-4557-b66a-c01e0211b530 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:14.157532 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd723f0e-ad81-453f-a86d-9c291d7796ee None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:14.158242 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd723f0e-ad81-453f-a86d-9c291d7796ee None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:14.158690 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd723f0e-ad81-453f-a86d-9c291d7796ee None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:14.161374 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 155/297] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:14.166492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:14.167548 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312eaf7-4e67-4fa7-98aa-1ce33827a42e None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:14.168011 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312eaf7-4e67-4fa7-98aa-1ce33827a42e None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:14.168620 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312eaf7-4e67-4fa7-98aa-1ce33827a42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:14.190739 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-0312eaf7-4e67-4fa7-98aa-1ce33827a42e None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:14.192301 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0312eaf7-4e67-4fa7-98aa-1ce33827a42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:14.280809 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104b952-309d-4557-b66a-c01e0211b530 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:14.285170 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 156/298] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:14.317947 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312eaf7-4e67-4fa7-98aa-1ce33827a42e None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:14.340816 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f063c-3027-41f0-90ea-0b3ce3906f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:14.341833 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f063c-3027-41f0-90ea-0b3ce3906f96 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:14.342453 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f063c-3027-41f0-90ea-0b3ce3906f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:14.342925 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f063c-3027-41f0-90ea-0b3ce3906f96 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:14.345389 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f063c-3027-41f0-90ea-0b3ce3906f96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:14.347613 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f063c-3027-41f0-90ea-0b3ce3906f96 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:14.350370 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 143/299] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:53:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:14.356589 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f063c-3027-41f0-90ea-0b3ce3906f96 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:14.497718 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312eaf7-4e67-4fa7-98aa-1ce33827a42e None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:14.516752 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:14.528896 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f063c-3027-41f0-90ea-0b3ce3906f96 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:14.550112 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:14.572533 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250335f6-7695-4730-89f4-499516e3f4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:14.573309 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250335f6-7695-4730-89f4-499516e3f4cf None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:14.573750 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250335f6-7695-4730-89f4-499516e3f4cf None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:14.574270 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250335f6-7695-4730-89f4-499516e3f4cf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:14.586745 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250335f6-7695-4730-89f4-499516e3f4cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:14.587986 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ed22e-c347-4bd6-a2da-da9ecc036e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:14.588809 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ed22e-c347-4bd6-a2da-da9ecc036e66 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:14.589229 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ed22e-c347-4bd6-a2da-da9ecc036e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:14.589640 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ed22e-c347-4bd6-a2da-da9ecc036e66 None admin] PATH_INFO: `/v3/groups` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:14.590441 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7ed22e-c347-4bd6-a2da-da9ecc036e66 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:14.592036 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7ed22e-c347-4bd6-a2da-da9ecc036e66 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:14.593156 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-250335f6-7695-4730-89f4-499516e3f4cf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:53:14.593156 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:14.593156 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:14.593156 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:14.593156 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:14.593156 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:14.593156 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:14.593156 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:14.593156 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:14.593156 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:14.593156 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:14.593156 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:14.593156 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:14.593156 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:14.593156 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:14.593156 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:14.594660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:14.594660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:14.594660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:14.594660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:14.594660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:14.594660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:14.594660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:14.594660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:14.594660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:14.594660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:14.594660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:14.594660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:14.594660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:14.594660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:14.594660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:14.594660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:14.596194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:14.596194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:14.596194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:14.596194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:14.596194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:14.596194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:14.596194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:14.596194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:14.596194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:14.596194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:14.596194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:53:14.596194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:14.596194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 157/300] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:53:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:14.602789 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250335f6-7695-4730-89f4-499516e3f4cf None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:14.632653 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 144/301] 10.222.0.45 () {60 vars in 1134 bytes} [Mon Aug 1 13:53:14 2022] POST /identity/v3/groups => generated 229 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:53:14.643631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16498157-8882-4b14-a025-a10a79ae45bb None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:14.643856 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16498157-8882-4b14-a025-a10a79ae45bb None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:14.644060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16498157-8882-4b14-a025-a10a79ae45bb None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:14.645601 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 145/302] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:53:14.652387 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4718c64-8b3a-4b3c-ae32-55eabe777a40 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:14.653156 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4718c64-8b3a-4b3c-ae32-55eabe777a40 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:14.653774 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4718c64-8b3a-4b3c-ae32-55eabe777a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:14.682959 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c4718c64-8b3a-4b3c-ae32-55eabe777a40 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:14.684496 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c4718c64-8b3a-4b3c-ae32-55eabe777a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:14.744740 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250335f6-7695-4730-89f4-499516e3f4cf None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:14.787239 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fce3d8-e2ce-41d5-9a41-9abfab24e395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:14.788312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fce3d8-e2ce-41d5-9a41-9abfab24e395 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:14.788817 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fce3d8-e2ce-41d5-9a41-9abfab24e395 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:14.789293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fce3d8-e2ce-41d5-9a41-9abfab24e395 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:14.790448 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fce3d8-e2ce-41d5-9a41-9abfab24e395 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:14.791000 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fce3d8-e2ce-41d5-9a41-9abfab24e395 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:14.792921 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fce3d8-e2ce-41d5-9a41-9abfab24e395 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:14.803885 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 158/303] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 13:53:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:53:14.810208 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fce3d8-e2ce-41d5-9a41-9abfab24e395 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:14.821408 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 146/304] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:14.843514 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4718c64-8b3a-4b3c-ae32-55eabe777a40 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:14.875465 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:14.946524 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fce3d8-e2ce-41d5-9a41-9abfab24e395 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:14.965167 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:14.977737 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4718c64-8b3a-4b3c-ae32-55eabe777a40 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:14.992796 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ddb1ee-07b0-4fdf-af4e-8ed672e3ea82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:14.994167 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddb1ee-07b0-4fdf-af4e-8ed672e3ea82 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:14.994686 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddb1ee-07b0-4fdf-af4e-8ed672e3ea82 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:14.995095 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddb1ee-07b0-4fdf-af4e-8ed672e3ea82 None admin] PATH_INFO: `/v3/users/66e04eaffcdb442896ecb3a531bf2c65` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:15.015410 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ddb1ee-07b0-4fdf-af4e-8ed672e3ea82 None admin] RBAC: Authorizing `identity:get_user(user_id=66e04eaffcdb442896ecb3a531bf2c65)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:15.018219 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ddb1ee-07b0-4fdf-af4e-8ed672e3ea82 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:15.020561 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 159/305] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:53:14 2022] GET /identity/v3/users/66e04eaffcdb442896ecb3a531bf2c65 => generated 293 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:15.026523 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ddb1ee-07b0-4fdf-af4e-8ed672e3ea82 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:15.028678 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715f4df-1992-49bb-bdb2-371c1bc9f675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:15.029994 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715f4df-1992-49bb-bdb2-371c1bc9f675 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:15.031047 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715f4df-1992-49bb-bdb2-371c1bc9f675 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:15.031731 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715f4df-1992-49bb-bdb2-371c1bc9f675 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:15.061131 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6715f4df-1992-49bb-bdb2-371c1bc9f675 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:15.076866 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-6715f4df-1992-49bb-bdb2-371c1bc9f675 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:53:15.076866 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:15.076866 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:15.076866 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:15.076866 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:15.076866 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:15.076866 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:15.076866 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:15.076866 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:15.076866 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:15.076866 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:15.076866 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:15.076866 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:15.076866 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:15.076866 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:15.076866 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:15.078804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:15.078804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:15.078804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:15.078804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:15.078804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:15.078804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:15.078804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:15.078804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:15.078804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:15.078804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:15.078804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:15.078804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:15.078804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:15.078804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:15.078804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:15.078804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:15.080231 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:15.080231 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:15.080231 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:15.080231 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:15.080231 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:15.080231 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:15.080231 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:15.080231 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:15.080231 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:15.080231 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:15.080231 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:53:15.080231 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:53:15.080231 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 147/306] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:53:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:15.085119 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715f4df-1992-49bb-bdb2-371c1bc9f675 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:15.208367 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ddb1ee-07b0-4fdf-af4e-8ed672e3ea82 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:15.211883 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:15.219844 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715f4df-1992-49bb-bdb2-371c1bc9f675 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:15.248803 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b53e5c3-b60f-4289-a002-cf9001602b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:15.249811 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53e5c3-b60f-4289-a002-cf9001602b15 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:15.250086 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53e5c3-b60f-4289-a002-cf9001602b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:15.250501 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53e5c3-b60f-4289-a002-cf9001602b15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:15.251439 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b53e5c3-b60f-4289-a002-cf9001602b15 None admin] RBAC: Adding query filter params (role.id=9db7d52dd9514261a5a52591fbf1feef, user.id=66e04eaffcdb442896ecb3a531bf2c65, scope.system=all) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:15.251928 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b53e5c3-b60f-4289-a002-cf9001602b15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:15.253793 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b53e5c3-b60f-4289-a002-cf9001602b15 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:15.259700 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 160/307] 10.222.0.45 () {56 vars in 1308 bytes} [Mon Aug 1 13:53:15 2022] GET /identity/v3/role_assignments?user.id=66e04eaffcdb442896ecb3a531bf2c65&scope.system=all&role.id=9db7d52dd9514261a5a52591fbf1feef => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:53:15.261311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b9fa4-0c63-47cc-b994-9b9c7f3d37c2 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:15.261585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b9fa4-0c63-47cc-b994-9b9c7f3d37c2 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:15.261851 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b9fa4-0c63-47cc-b994-9b9c7f3d37c2 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:15.263146 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 161/308] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:15.270232 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0adb8-3676-43f0-ad88-5561df2f0cf4 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:15.270748 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0adb8-3676-43f0-ad88-5561df2f0cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:15.271310 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0adb8-3676-43f0-ad88-5561df2f0cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:15.274714 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edddc68-e646-4235-9d92-f6214d6a7a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:15.275456 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edddc68-e646-4235-9d92-f6214d6a7a81 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:15.275809 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edddc68-e646-4235-9d92-f6214d6a7a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:15.276146 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edddc68-e646-4235-9d92-f6214d6a7a81 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:15.277177 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edddc68-e646-4235-9d92-f6214d6a7a81 None admin] RBAC: Adding query filter params (name=member) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:15.277676 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edddc68-e646-4235-9d92-f6214d6a7a81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:15.279664 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edddc68-e646-4235-9d92-f6214d6a7a81 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:15.293784 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-f6a0adb8-3676-43f0-ad88-5561df2f0cf4 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:15.295212 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 148/309] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 13:53:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:15.295510 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-f6a0adb8-3676-43f0-ad88-5561df2f0cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:15.302721 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edddc68-e646-4235-9d92-f6214d6a7a81 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:15.320142 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:15.381446 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 162/310] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:15.404777 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0adb8-3676-43f0-ad88-5561df2f0cf4 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:15.449736 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edddc68-e646-4235-9d92-f6214d6a7a81 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:15.496322 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295bd99-3034-45bd-b5fd-5697800bee38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:15.496970 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295bd99-3034-45bd-b5fd-5697800bee38 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:15.497274 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295bd99-3034-45bd-b5fd-5697800bee38 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:15.497552 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295bd99-3034-45bd-b5fd-5697800bee38 None admin] PATH_INFO: `/v3/users/d6d0746eb63c46ffbcc1ec5ed80e20e5` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:15.498774 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2295bd99-3034-45bd-b5fd-5697800bee38 None admin] RBAC: Authorizing `identity:get_user(user_id=d6d0746eb63c46ffbcc1ec5ed80e20e5)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:15.500663 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2295bd99-3034-45bd-b5fd-5697800bee38 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:15.502763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 149/311] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:53:15 2022] GET /identity/v3/users/d6d0746eb63c46ffbcc1ec5ed80e20e5 => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:15.509765 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295bd99-3034-45bd-b5fd-5697800bee38 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:15.583889 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:15.596986 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0adb8-3676-43f0-ad88-5561df2f0cf4 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:15.627442 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:15.657426 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295bd99-3034-45bd-b5fd-5697800bee38 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:15.663628 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f434207-313c-4fdc-a0f5-95ae3b5d900a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:15.664650 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f434207-313c-4fdc-a0f5-95ae3b5d900a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:15.665149 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f434207-313c-4fdc-a0f5-95ae3b5d900a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:15.665636 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f434207-313c-4fdc-a0f5-95ae3b5d900a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:15.683158 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f434207-313c-4fdc-a0f5-95ae3b5d900a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:15.690832 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-2f434207-313c-4fdc-a0f5-95ae3b5d900a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:53:15.690832 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:15.690832 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:15.690832 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:15.690832 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:15.690832 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:15.690832 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:15.690832 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:15.690832 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:15.690832 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:15.690832 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:15.690832 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:15.690832 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:15.690832 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:15.690832 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:15.690832 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:15.691719 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:15.691719 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:15.691719 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:15.691719 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:15.691719 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:15.691719 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:15.691719 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:15.691719 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:15.691719 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:15.691719 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:15.691719 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:15.691719 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:15.691719 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:15.691719 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:15.691719 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:15.691719 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:15.692569 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:15.692569 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:15.692569 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:15.692569 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:15.692569 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:15.692569 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:15.692569 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:15.692569 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:15.692569 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:15.692569 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:15.692569 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:53:15.692569 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:15.693925 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 163/312] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:53:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:53:15.703203 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f434207-313c-4fdc-a0f5-95ae3b5d900a None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:15.703422 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a079a8-af3b-4704-b342-e26095007ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:15.703422 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a079a8-af3b-4704-b342-e26095007ed2 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:15.703422 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a079a8-af3b-4704-b342-e26095007ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:15.703422 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a079a8-af3b-4704-b342-e26095007ed2 None admin] PATH_INFO: `/v3/system/users/d6d0746eb63c46ffbcc1ec5ed80e20e5/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:15.704775 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a079a8-af3b-4704-b342-e26095007ed2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d6d0746eb63c46ffbcc1ec5ed80e20e5, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:15.706286 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a079a8-af3b-4704-b342-e26095007ed2 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:15.718530 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 150/313] 10.222.0.45 () {58 vars in 1334 bytes} [Mon Aug 1 13:53:15 2022] PUT /identity/v3/system/users/d6d0746eb63c46ffbcc1ec5ed80e20e5/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:53:15.830101 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f434207-313c-4fdc-a0f5-95ae3b5d900a None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:15.871667 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44f4016-d8c6-4f6c-ba43-c0b5702f50ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:15.872686 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44f4016-d8c6-4f6c-ba43-c0b5702f50ef None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:15.873226 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44f4016-d8c6-4f6c-ba43-c0b5702f50ef None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:15.873678 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44f4016-d8c6-4f6c-ba43-c0b5702f50ef None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:15.874693 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44f4016-d8c6-4f6c-ba43-c0b5702f50ef None admin] RBAC: Adding query filter params (name=reader) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:15.875396 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44f4016-d8c6-4f6c-ba43-c0b5702f50ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:15.877328 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44f4016-d8c6-4f6c-ba43-c0b5702f50ef None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:15.892411 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 164/314] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 13:53:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:15.896702 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a079a8-af3b-4704-b342-e26095007ed2 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:16.018406 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a079a8-af3b-4704-b342-e26095007ed2 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:16.057352 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63638fb-59ef-46b0-ab8a-80de3435ecff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:16.058027 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63638fb-59ef-46b0-ab8a-80de3435ecff None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:16.058309 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63638fb-59ef-46b0-ab8a-80de3435ecff None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:16.058588 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63638fb-59ef-46b0-ab8a-80de3435ecff None admin] PATH_INFO: `/v3/users/ed7c48945e7a45a68a0e4156685ed750` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:16.060056 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63638fb-59ef-46b0-ab8a-80de3435ecff None admin] RBAC: Authorizing `identity:get_user(user_id=ed7c48945e7a45a68a0e4156685ed750)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:16.063046 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63638fb-59ef-46b0-ab8a-80de3435ecff None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:16.066570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 151/315] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:53:15 2022] GET /identity/v3/users/ed7c48945e7a45a68a0e4156685ed750 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:16.072165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44f4016-d8c6-4f6c-ba43-c0b5702f50ef None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:16.105055 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:16.181868 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44f4016-d8c6-4f6c-ba43-c0b5702f50ef None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:16.218921 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aba253-654a-445f-a568-5902be2ded7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:16.220702 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aba253-654a-445f-a568-5902be2ded7c None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:16.221240 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aba253-654a-445f-a568-5902be2ded7c None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:16.221708 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aba253-654a-445f-a568-5902be2ded7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:16.312103 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aba253-654a-445f-a568-5902be2ded7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:16.313652 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aba253-654a-445f-a568-5902be2ded7c None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:16.342234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:16.442554 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 165/316] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:53:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:53:16.459101 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63638fb-59ef-46b0-ab8a-80de3435ecff None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:16.478681 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:16.634982 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63638fb-59ef-46b0-ab8a-80de3435ecff None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:16.692129 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262411a-22d6-4156-b3a7-89f5daa87eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:16.692942 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262411a-22d6-4156-b3a7-89f5daa87eff None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:16.693171 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262411a-22d6-4156-b3a7-89f5daa87eff None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:16.693448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262411a-22d6-4156-b3a7-89f5daa87eff None admin] PATH_INFO: `/v3/projects/7489ab03e4a8475cab57be63c14ec0c0` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:16.694780 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8262411a-22d6-4156-b3a7-89f5daa87eff None admin] RBAC: Authorizing `identity:get_project(project_id=7489ab03e4a8475cab57be63c14ec0c0)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:16.696298 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8262411a-22d6-4156-b3a7-89f5daa87eff None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:16.703492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 152/317] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:53:16 2022] GET /identity/v3/projects/7489ab03e4a8475cab57be63c14ec0c0 => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:16.705076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aba253-654a-445f-a568-5902be2ded7c None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:16.856500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aba253-654a-445f-a568-5902be2ded7c None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:16.884208 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:16.900350 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f3e4a-cb5a-46ab-83d9-c14fe94c9a39 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:16.900752 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f3e4a-cb5a-46ab-83d9-c14fe94c9a39 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:16.901122 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f3e4a-cb5a-46ab-83d9-c14fe94c9a39 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:16.902906 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 153/318] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:53:16.905514 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c34a5b-f52a-4c23-a103-51a145a3bf4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:16.906260 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c34a5b-f52a-4c23-a103-51a145a3bf4a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:16.906538 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c34a5b-f52a-4c23-a103-51a145a3bf4a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:16.906877 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c34a5b-f52a-4c23-a103-51a145a3bf4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:16.908437 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c34a5b-f52a-4c23-a103-51a145a3bf4a None admin] RBAC: Adding query filter params (role.id=9db7d52dd9514261a5a52591fbf1feef, scope.project.id=7489ab03e4a8475cab57be63c14ec0c0, user.id=ed7c48945e7a45a68a0e4156685ed750) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:16.908804 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c34a5b-f52a-4c23-a103-51a145a3bf4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:16.909163 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd879cb8-d8c7-48f8-9a6f-2e59c0826d77 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:16.909570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd879cb8-d8c7-48f8-9a6f-2e59c0826d77 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:16.910015 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd879cb8-d8c7-48f8-9a6f-2e59c0826d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:16.910318 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c34a5b-f52a-4c23-a103-51a145a3bf4a None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:16.917691 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 166/319] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:53:16 2022] GET /identity/v3/role_assignments?user.id=ed7c48945e7a45a68a0e4156685ed750&scope.project.id=7489ab03e4a8475cab57be63c14ec0c0&role.id=9db7d52dd9514261a5a52591fbf1feef => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:16.944690 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-dd879cb8-d8c7-48f8-9a6f-2e59c0826d77 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:16.945858 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-dd879cb8-d8c7-48f8-9a6f-2e59c0826d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:17.043901 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 154/320] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:53:17.082046 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c34a5b-f52a-4c23-a103-51a145a3bf4a None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:17.236235 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c34a5b-f52a-4c23-a103-51a145a3bf4a None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:17.283291 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7307968b-5ff9-4bd4-abee-1cd9d1e8fefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:17.283973 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307968b-5ff9-4bd4-abee-1cd9d1e8fefd None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:17.284252 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307968b-5ff9-4bd4-abee-1cd9d1e8fefd None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:17.284558 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307968b-5ff9-4bd4-abee-1cd9d1e8fefd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:17.285857 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7307968b-5ff9-4bd4-abee-1cd9d1e8fefd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:17.287526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7307968b-5ff9-4bd4-abee-1cd9d1e8fefd None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:17.289902 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 167/321] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:53:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:17.295733 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd879cb8-d8c7-48f8-9a6f-2e59c0826d77 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:17.305376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:17.444379 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd879cb8-d8c7-48f8-9a6f-2e59c0826d77 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:17.504910 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2aec2-4a13-4e7a-ba52-5c67e07b83b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:17.505737 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2aec2-4a13-4e7a-ba52-5c67e07b83b8 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:17.506209 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2aec2-4a13-4e7a-ba52-5c67e07b83b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:17.506656 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2aec2-4a13-4e7a-ba52-5c67e07b83b8 None admin] PATH_INFO: `/v3/groups` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:17.507685 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf2aec2-4a13-4e7a-ba52-5c67e07b83b8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:17.510240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf2aec2-4a13-4e7a-ba52-5c67e07b83b8 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:17.542497 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 155/322] 10.222.0.45 () {60 vars in 1134 bytes} [Mon Aug 1 13:53:17 2022] POST /identity/v3/groups => generated 226 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:53:18.142039 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4aca43-535a-4708-8021-096dda21b2bf None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:18.143230 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4aca43-535a-4708-8021-096dda21b2bf None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:18.143362 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4aca43-535a-4708-8021-096dda21b2bf None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:18.146361 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 168/323] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:18.155590 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30958468-e1c9-4277-8084-46f912cf0703 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:18.156480 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30958468-e1c9-4277-8084-46f912cf0703 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:18.156757 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30958468-e1c9-4277-8084-46f912cf0703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:18.190310 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-30958468-e1c9-4277-8084-46f912cf0703 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:18.192436 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-30958468-e1c9-4277-8084-46f912cf0703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:18.286039 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5975ab-7647-4341-b2c9-2c91f4e20770 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:18.286502 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5975ab-7647-4341-b2c9-2c91f4e20770 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:18.286966 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5975ab-7647-4341-b2c9-2c91f4e20770 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:18.289470 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 169/324] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:53:18.298627 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c081c-435a-4167-a272-35954fd49e24 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:18.300052 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c081c-435a-4167-a272-35954fd49e24 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:18.300052 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c081c-435a-4167-a272-35954fd49e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:18.305196 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:18.316007 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 156/325] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:18.331490 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-f62c081c-435a-4167-a272-35954fd49e24 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:18.332754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-f62c081c-435a-4167-a272-35954fd49e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:18.357194 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30958468-e1c9-4277-8084-46f912cf0703 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:18.422548 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:18.464792 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 170/326] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:18.507773 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62c081c-435a-4167-a272-35954fd49e24 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:18.511116 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30958468-e1c9-4277-8084-46f912cf0703 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:18.560953 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa774b8e-d845-48b9-a0e2-3becf3792bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:18.562022 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa774b8e-d845-48b9-a0e2-3becf3792bac None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:18.562487 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa774b8e-d845-48b9-a0e2-3becf3792bac None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:18.562939 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa774b8e-d845-48b9-a0e2-3becf3792bac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:18.597663 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa774b8e-d845-48b9-a0e2-3becf3792bac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:18.608264 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-aa774b8e-d845-48b9-a0e2-3becf3792bac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:53:18.608264 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:18.608264 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:18.608264 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:18.608264 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:18.608264 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:18.608264 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:18.608264 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:18.608264 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:18.608264 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:18.608264 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:18.608264 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:18.608264 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:18.608264 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:18.608264 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:18.608264 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:18.609825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:18.609825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:18.609825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:18.609825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:18.609825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:18.609825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:18.609825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:18.609825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:18.609825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:18.609825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:18.609825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:18.609825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:18.609825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:18.609825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:18.609825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:18.609825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:18.610721 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:18.610721 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:18.610721 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:18.610721 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:18.610721 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:18.610721 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:18.610721 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:18.610721 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:18.610721 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:18.610721 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:18.610721 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:53:18.610721 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:53:18.615837 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 157/327] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:53:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:53:18.640834 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa774b8e-d845-48b9-a0e2-3becf3792bac None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:18.824818 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62c081c-435a-4167-a272-35954fd49e24 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:18.828022 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:18.921224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b0012-24c1-4afd-8ac1-0a63bd347f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:18.922428 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b0012-24c1-4afd-8ac1-0a63bd347f62 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:18.922959 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b0012-24c1-4afd-8ac1-0a63bd347f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:18.923506 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b0012-24c1-4afd-8ac1-0a63bd347f62 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:18.952357 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b0012-24c1-4afd-8ac1-0a63bd347f62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:18.954584 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa774b8e-d845-48b9-a0e2-3becf3792bac None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:18.968302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-ad6b0012-24c1-4afd-8ac1-0a63bd347f62 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:53:18.968302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:18.968302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:18.968302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:18.968302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:18.968302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:18.968302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:18.968302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:18.968302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:18.968302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:18.968302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:18.968302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:18.968302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:18.968302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:18.968302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:18.968302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:18.969304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:18.969304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:18.969304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:18.969304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:18.969304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:18.969304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:18.969304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:18.969304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:18.969304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:18.969304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:18.969304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:18.969304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:18.969304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:18.969304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:18.969304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:18.969304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:18.970106 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:18.970106 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:18.970106 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:18.970106 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:18.970106 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:18.970106 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:18.970106 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:18.970106 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:18.970106 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:18.970106 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:18.970106 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:53:18.970106 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:18.971546 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 171/328] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:53:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 465 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:18.982038 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b0012-24c1-4afd-8ac1-0a63bd347f62 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:19.026187 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cff82-0a4a-4bc1-8387-f8955ebc62ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:19.026870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cff82-0a4a-4bc1-8387-f8955ebc62ab None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:19.027220 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cff82-0a4a-4bc1-8387-f8955ebc62ab None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:19.027491 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cff82-0a4a-4bc1-8387-f8955ebc62ab None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:19.028675 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832cff82-0a4a-4bc1-8387-f8955ebc62ab None admin] RBAC: Adding query filter params (name=reader) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:19.029021 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832cff82-0a4a-4bc1-8387-f8955ebc62ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:19.030436 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832cff82-0a4a-4bc1-8387-f8955ebc62ab None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:19.045792 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 158/329] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 13:53:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:19.052363 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cff82-0a4a-4bc1-8387-f8955ebc62ab None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:19.093085 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:19.131821 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b0012-24c1-4afd-8ac1-0a63bd347f62 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:19.197023 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55699648-8d33-43e6-a4cd-f7de3fffbf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:19.198469 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55699648-8d33-43e6-a4cd-f7de3fffbf0a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:19.198971 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55699648-8d33-43e6-a4cd-f7de3fffbf0a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:19.199441 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55699648-8d33-43e6-a4cd-f7de3fffbf0a None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:19.200695 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55699648-8d33-43e6-a4cd-f7de3fffbf0a None admin] RBAC: Adding query filter params (name=member) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:19.201239 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55699648-8d33-43e6-a4cd-f7de3fffbf0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:19.202978 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55699648-8d33-43e6-a4cd-f7de3fffbf0a None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:19.220008 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 172/330] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 13:53:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:19.227014 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55699648-8d33-43e6-a4cd-f7de3fffbf0a None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:19.269994 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cff82-0a4a-4bc1-8387-f8955ebc62ab None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:19.311286 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:19.350272 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200cdcd-d0c5-47be-9fc9-c58d6867a680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:19.351556 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200cdcd-d0c5-47be-9fc9-c58d6867a680 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:19.352174 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200cdcd-d0c5-47be-9fc9-c58d6867a680 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:19.352730 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200cdcd-d0c5-47be-9fc9-c58d6867a680 None admin] PATH_INFO: `/v3/users/66e04eaffcdb442896ecb3a531bf2c65` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:19.356375 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200cdcd-d0c5-47be-9fc9-c58d6867a680 None admin] RBAC: Authorizing `identity:get_user(user_id=66e04eaffcdb442896ecb3a531bf2c65)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:19.360019 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200cdcd-d0c5-47be-9fc9-c58d6867a680 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:19.363572 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 159/331] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:53:19 2022] GET /identity/v3/users/66e04eaffcdb442896ecb3a531bf2c65 => generated 293 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:53:19.373550 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200cdcd-d0c5-47be-9fc9-c58d6867a680 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:19.522453 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55699648-8d33-43e6-a4cd-f7de3fffbf0a None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:19.526489 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:19.609113 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4a291-e467-4abf-a964-6e35987c343e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:19.610280 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4a291-e467-4abf-a964-6e35987c343e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:19.612092 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4a291-e467-4abf-a964-6e35987c343e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:19.612521 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4a291-e467-4abf-a964-6e35987c343e None admin] PATH_INFO: `/v3/users/d6d0746eb63c46ffbcc1ec5ed80e20e5` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:19.615036 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb4a291-e467-4abf-a964-6e35987c343e None admin] RBAC: Authorizing `identity:get_user(user_id=d6d0746eb63c46ffbcc1ec5ed80e20e5)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:19.616909 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb4a291-e467-4abf-a964-6e35987c343e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:19.625823 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 173/332] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:53:19 2022] GET /identity/v3/users/d6d0746eb63c46ffbcc1ec5ed80e20e5 => generated 293 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:53:19.630168 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200cdcd-d0c5-47be-9fc9-c58d6867a680 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:19.644143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4a291-e467-4abf-a964-6e35987c343e None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:19.712682 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed348d2-35c2-4388-aaf4-c77e093adec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:19.713403 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed348d2-35c2-4388-aaf4-c77e093adec5 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:19.713691 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed348d2-35c2-4388-aaf4-c77e093adec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:19.713965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed348d2-35c2-4388-aaf4-c77e093adec5 None admin] PATH_INFO: `/v3/system/users/66e04eaffcdb442896ecb3a531bf2c65/roles/9db7d52dd9514261a5a52591fbf1feef` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:19.717133 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed348d2-35c2-4388-aaf4-c77e093adec5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=66e04eaffcdb442896ecb3a531bf2c65, role_id=9db7d52dd9514261a5a52591fbf1feef)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:19.719682 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed348d2-35c2-4388-aaf4-c77e093adec5 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:19.737186 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 160/333] 10.222.0.45 () {58 vars in 1334 bytes} [Mon Aug 1 13:53:19 2022] PUT /identity/v3/system/users/66e04eaffcdb442896ecb3a531bf2c65/roles/9db7d52dd9514261a5a52591fbf1feef => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:53:19.875363 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4a291-e467-4abf-a964-6e35987c343e None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:19.943231 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579dc4b5-9c3e-4f8f-8e0e-3fe8dab6b91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:19.944307 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579dc4b5-9c3e-4f8f-8e0e-3fe8dab6b91b None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:19.944888 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579dc4b5-9c3e-4f8f-8e0e-3fe8dab6b91b None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:19.945433 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579dc4b5-9c3e-4f8f-8e0e-3fe8dab6b91b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:19.947004 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579dc4b5-9c3e-4f8f-8e0e-3fe8dab6b91b None admin] RBAC: Adding query filter params (role.id=75aff8cb4cf748a2adbd6c3f141fe8ac, user.id=d6d0746eb63c46ffbcc1ec5ed80e20e5, scope.system=all) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:19.947545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579dc4b5-9c3e-4f8f-8e0e-3fe8dab6b91b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:19.950223 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579dc4b5-9c3e-4f8f-8e0e-3fe8dab6b91b None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:19.973581 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 174/334] 10.222.0.45 () {56 vars in 1308 bytes} [Mon Aug 1 13:53:19 2022] GET /identity/v3/role_assignments?user.id=d6d0746eb63c46ffbcc1ec5ed80e20e5&scope.system=all&role.id=75aff8cb4cf748a2adbd6c3f141fe8ac => generated 510 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:53:21.887360 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162c4c2-f8a0-4d80-a573-5fa0a8adb4bb None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:21.888917 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162c4c2-f8a0-4d80-a573-5fa0a8adb4bb None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:21.889714 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162c4c2-f8a0-4d80-a573-5fa0a8adb4bb None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:21.893369 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 161/335] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:21.901498 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62182219-7bf1-4677-9459-c343d3dba361 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:21.902526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62182219-7bf1-4677-9459-c343d3dba361 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:21.903101 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62182219-7bf1-4677-9459-c343d3dba361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:21.924558 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-62182219-7bf1-4677-9459-c343d3dba361 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:21.926138 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-62182219-7bf1-4677-9459-c343d3dba361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:21.965771 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:22.061263 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 175/336] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:53:22.123890 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162c4c2-f8a0-4d80-a573-5fa0a8adb4bb None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:22.209531 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:22.271353 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3dffbd-ce94-46e8-8780-d27a83f28acd None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:22.271926 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3dffbd-ce94-46e8-8780-d27a83f28acd None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:22.272523 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3dffbd-ce94-46e8-8780-d27a83f28acd None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:22.275297 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 176/337] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:22.284796 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c9798b-e848-4b25-8e8e-9c0f67ca73f6 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:22.285951 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c9798b-e848-4b25-8e8e-9c0f67ca73f6 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:22.286623 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c9798b-e848-4b25-8e8e-9c0f67ca73f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:22.328571 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-04c9798b-e848-4b25-8e8e-9c0f67ca73f6 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:22.331566 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-04c9798b-e848-4b25-8e8e-9c0f67ca73f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:22.341405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162c4c2-f8a0-4d80-a573-5fa0a8adb4bb None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:22.467771 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 177/338] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:22.473191 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c4680b-0dfd-49ac-9f24-b6c69f91f917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:22.474235 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4680b-0dfd-49ac-9f24-b6c69f91f917 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:22.474720 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4680b-0dfd-49ac-9f24-b6c69f91f917 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:22.475198 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4680b-0dfd-49ac-9f24-b6c69f91f917 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:22.518226 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c4680b-0dfd-49ac-9f24-b6c69f91f917 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:22.527637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-e0c4680b-0dfd-49ac-9f24-b6c69f91f917 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:53:22.527637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:22.527637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:22.527637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:22.527637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:22.527637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:22.527637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:22.527637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:22.527637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:22.527637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:22.527637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:22.527637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:22.527637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:22.527637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:22.527637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:22.527637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:22.528874 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:22.528874 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:22.528874 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:22.528874 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:22.528874 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:22.528874 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:22.528874 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:22.528874 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:22.528874 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:22.528874 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:22.528874 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:22.528874 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:22.528874 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:22.528874 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:22.528874 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:22.528874 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:22.529887 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:22.529887 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:22.529887 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:22.529887 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:22.529887 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:22.529887 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:22.529887 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:22.529887 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:22.529887 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:22.529887 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:22.529887 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:53:22.529887 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:53:22.535709 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 162/339] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:53:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 407 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:22.539256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c9798b-e848-4b25-8e8e-9c0f67ca73f6 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:22.555210 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c4680b-0dfd-49ac-9f24-b6c69f91f917 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:22.724656 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:22.733048 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c9798b-e848-4b25-8e8e-9c0f67ca73f6 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:22.771337 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c4680b-0dfd-49ac-9f24-b6c69f91f917 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:22.804699 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:22.813425 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86fc396-5c56-4f81-a01d-bcd2412a4d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:22.814465 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fc396-5c56-4f81-a01d-bcd2412a4d55 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:22.814955 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fc396-5c56-4f81-a01d-bcd2412a4d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:22.815464 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fc396-5c56-4f81-a01d-bcd2412a4d55 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:22.817702 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86fc396-5c56-4f81-a01d-bcd2412a4d55 None admin] RBAC: Adding query filter params (name=member) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:22.818519 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86fc396-5c56-4f81-a01d-bcd2412a4d55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:22.821315 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86fc396-5c56-4f81-a01d-bcd2412a4d55 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:22.840390 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 178/340] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 13:53:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:22.843464 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0202cc4-7d41-4064-ac97-ea091a63717c None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:22.843863 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0202cc4-7d41-4064-ac97-ea091a63717c None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:22.849374 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0202cc4-7d41-4064-ac97-ea091a63717c None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:22.852168 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 179/341] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:22 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:22.865718 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0202cc4-7d41-4064-ac97-ea091a63717c None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:22.881029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22f5aa-9d70-4e29-8131-4aec16e4797b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:22.881697 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22f5aa-9d70-4e29-8131-4aec16e4797b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:22.881974 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22f5aa-9d70-4e29-8131-4aec16e4797b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:22.882247 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22f5aa-9d70-4e29-8131-4aec16e4797b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:22.913674 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a22f5aa-9d70-4e29-8131-4aec16e4797b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:22.918754 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-7a22f5aa-9d70-4e29-8131-4aec16e4797b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:53:22.918754 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:22.918754 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:22.918754 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:22.918754 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:22.918754 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:22.918754 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:22.918754 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:22.918754 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:22.918754 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:22.918754 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:22.918754 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:22.918754 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:22.918754 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:22.918754 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:22.918754 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:22.919672 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:22.919672 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:22.919672 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:22.919672 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:22.919672 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:22.919672 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:22.919672 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:22.919672 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:22.919672 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:22.919672 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:22.919672 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:22.919672 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:22.919672 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:22.919672 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:22.919672 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:22.919672 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:22.920523 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:22.920523 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:22.920523 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:22.920523 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:22.920523 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:22.920523 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:22.920523 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:22.920523 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:22.920523 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:22.920523 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:22.920523 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:53:22.920523 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:53:22.924413 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 163/342] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 13:53:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:22.925813 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c4fa2-b7c3-4958-a5b5-10c238aaee35 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:22.926099 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c4fa2-b7c3-4958-a5b5-10c238aaee35 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:22.926374 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c4fa2-b7c3-4958-a5b5-10c238aaee35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:22.969477 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-bc4c4fa2-b7c3-4958-a5b5-10c238aaee35 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:22.970692 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-bc4c4fa2-b7c3-4958-a5b5-10c238aaee35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:23.074595 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0202cc4-7d41-4064-ac97-ea091a63717c None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:23.148178 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981fdc8e-df19-4ce8-8c5a-a29c33793f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:23.149288 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981fdc8e-df19-4ce8-8c5a-a29c33793f17 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:23.149800 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981fdc8e-df19-4ce8-8c5a-a29c33793f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:23.150269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981fdc8e-df19-4ce8-8c5a-a29c33793f17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:23.176492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 164/343] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:23.177796 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c4fa2-b7c3-4958-a5b5-10c238aaee35 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:23.200229 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:23.314474 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981fdc8e-df19-4ce8-8c5a-a29c33793f17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:23.316380 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981fdc8e-df19-4ce8-8c5a-a29c33793f17 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:23.318465 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:23.323345 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c4fa2-b7c3-4958-a5b5-10c238aaee35 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:23.373512 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07eb04b-6b0f-4e36-83dc-299bdb49eecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:23.374525 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07eb04b-6b0f-4e36-83dc-299bdb49eecd None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:23.375011 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07eb04b-6b0f-4e36-83dc-299bdb49eecd None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:23.375486 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07eb04b-6b0f-4e36-83dc-299bdb49eecd None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:23.376439 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07eb04b-6b0f-4e36-83dc-299bdb49eecd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:23.376932 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07eb04b-6b0f-4e36-83dc-299bdb49eecd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:23.378522 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07eb04b-6b0f-4e36-83dc-299bdb49eecd None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:23.389892 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 165/344] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:53:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:23.391570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c7e13c-246b-4adb-90cb-e3e27ceb14e6 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:23.392143 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c7e13c-246b-4adb-90cb-e3e27ceb14e6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:23.392767 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c7e13c-246b-4adb-90cb-e3e27ceb14e6 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:23.394651 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 166/345] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:23.395600 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c7e13c-246b-4adb-90cb-e3e27ceb14e6 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:23.542037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 180/346] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:53:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 676 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:53:23.545792 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981fdc8e-df19-4ce8-8c5a-a29c33793f17 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:23.576216 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c7e13c-246b-4adb-90cb-e3e27ceb14e6 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:23.623005 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf26dc-9c61-4f0c-af10-b74830e0f058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:23.624103 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf26dc-9c61-4f0c-af10-b74830e0f058 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:23.625065 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf26dc-9c61-4f0c-af10-b74830e0f058 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:23.625614 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf26dc-9c61-4f0c-af10-b74830e0f058 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:23.639251 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baf26dc-9c61-4f0c-af10-b74830e0f058 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:23.646524 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-9baf26dc-9c61-4f0c-af10-b74830e0f058 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:53:23.646524 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:23.646524 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:23.646524 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:23.646524 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:23.646524 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:23.646524 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:23.646524 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:23.646524 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:23.646524 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:23.646524 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:23.646524 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:23.646524 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:23.646524 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:23.646524 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:23.646524 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:23.647552 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:23.647552 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:23.647552 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:23.647552 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:23.647552 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:23.647552 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:23.647552 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:23.647552 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:23.647552 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:23.647552 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:23.647552 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:23.647552 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:23.647552 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:23.647552 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:23.647552 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:23.647552 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:23.648488 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:23.648488 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:23.648488 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:23.648488 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:23.648488 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:23.648488 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:23.648488 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:23.648488 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:23.648488 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:23.648488 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:23.648488 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:53:23.648488 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:53:23.651016 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 167/347] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:53:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:23.652552 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937f891-9206-4116-bda2-9d646615233a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:23.653116 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937f891-9206-4116-bda2-9d646615233a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:23.653577 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937f891-9206-4116-bda2-9d646615233a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:23.675886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-9937f891-9206-4116-bda2-9d646615233a None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:23.678335 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9937f891-9206-4116-bda2-9d646615233a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:23.678472 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:23.740328 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981fdc8e-df19-4ce8-8c5a-a29c33793f17 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:23.742605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:23.785208 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff098dd7-f230-426e-9780-6b548a48e315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:23.787561 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff098dd7-f230-426e-9780-6b548a48e315 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:23.788087 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff098dd7-f230-426e-9780-6b548a48e315 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:23.788587 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff098dd7-f230-426e-9780-6b548a48e315 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:23.837364 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 168/348] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:23.838747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937f891-9206-4116-bda2-9d646615233a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:23.900017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff098dd7-f230-426e-9780-6b548a48e315 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:23.903088 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff098dd7-f230-426e-9780-6b548a48e315 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:23.978848 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:23.996574 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937f891-9206-4116-bda2-9d646615233a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:24.056314 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 181/349] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:53:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:53:24.058992 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea68ecd-f088-4469-bbd0-6bda85620078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:24.059211 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff098dd7-f230-426e-9780-6b548a48e315 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:24.060434 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea68ecd-f088-4469-bbd0-6bda85620078 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:24.060872 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea68ecd-f088-4469-bbd0-6bda85620078 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:24.061476 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea68ecd-f088-4469-bbd0-6bda85620078 None admin] PATH_INFO: `/v3/projects/c18388255f054822b773ec88708292da` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:24.063200 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea68ecd-f088-4469-bbd0-6bda85620078 None admin] RBAC: Authorizing `identity:get_project(project_id=c18388255f054822b773ec88708292da)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:24.065343 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea68ecd-f088-4469-bbd0-6bda85620078 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:24.068248 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 169/350] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:53:23 2022] GET /identity/v3/projects/c18388255f054822b773ec88708292da => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:24.070253 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6bde8-c2b3-45b9-99e1-efcb0f399285 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:24.070688 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6bde8-c2b3-45b9-99e1-efcb0f399285 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:24.071119 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6bde8-c2b3-45b9-99e1-efcb0f399285 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:24.073605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 170/351] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:24.074932 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6bde8-c2b3-45b9-99e1-efcb0f399285 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:24.102973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:24.221703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6bde8-c2b3-45b9-99e1-efcb0f399285 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:24.235460 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff098dd7-f230-426e-9780-6b548a48e315 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:24.276007 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b12a71-4092-4962-9f53-6b03927e8755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:24.276839 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b12a71-4092-4962-9f53-6b03927e8755 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:24.277116 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b12a71-4092-4962-9f53-6b03927e8755 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:24.277389 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b12a71-4092-4962-9f53-6b03927e8755 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:24.295070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b12a71-4092-4962-9f53-6b03927e8755 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:24.297036 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08193dfe-6af1-4915-9164-a7f25dc74655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:24.297816 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08193dfe-6af1-4915-9164-a7f25dc74655 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:24.298341 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08193dfe-6af1-4915-9164-a7f25dc74655 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:24.298717 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08193dfe-6af1-4915-9164-a7f25dc74655 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:24.298864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-31b12a71-4092-4962-9f53-6b03927e8755 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:53:24.298864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:24.298864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:24.298864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:24.298864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:24.298864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:24.298864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:24.298864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:24.298864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:24.298864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:24.298864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:24.298864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:24.298864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:24.298864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:24.298864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:24.298864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:24.300921 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:24.300921 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:24.300921 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:24.300921 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:24.300921 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:24.300921 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:24.300921 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:24.300921 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:24.300921 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:24.300921 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:24.300921 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:24.300921 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:24.300921 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:24.300921 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:24.300921 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:24.300921 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:24.302605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:24.302605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:24.302605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:24.302605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:24.302605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:24.302605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:24.302605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:24.302605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:24.302605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:24.302605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:24.302605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:53:24.302605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:53:24.302605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 171/352] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:53:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:24.309697 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08193dfe-6af1-4915-9164-a7f25dc74655 None admin] RBAC: Adding query filter params (name=member) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:24.309697 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08193dfe-6af1-4915-9164-a7f25dc74655 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:24.309958 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533f1e9-daca-49e6-8ddf-9d8090a1a360 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:24.309958 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533f1e9-daca-49e6-8ddf-9d8090a1a360 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:24.309958 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533f1e9-daca-49e6-8ddf-9d8090a1a360 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:24.309958 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 172/353] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:24.309958 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9533f1e9-daca-49e6-8ddf-9d8090a1a360 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:24.310545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08193dfe-6af1-4915-9164-a7f25dc74655 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:24.321166 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 182/354] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 13:53:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:24.323787 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56e8a1-eda8-4b17-8df8-a537f56aff23 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:24.324299 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56e8a1-eda8-4b17-8df8-a537f56aff23 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:24.324786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56e8a1-eda8-4b17-8df8-a537f56aff23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:24.351737 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-dc56e8a1-eda8-4b17-8df8-a537f56aff23 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:24.352148 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-dc56e8a1-eda8-4b17-8df8-a537f56aff23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:24.370329 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:24.428458 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:24.459584 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9533f1e9-daca-49e6-8ddf-9d8090a1a360 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:24.466472 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 183/355] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:24.467867 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56e8a1-eda8-4b17-8df8-a537f56aff23 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:24.527304 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f5ef6-074c-46f5-a2c5-80c80a698837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:24.528206 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f5ef6-074c-46f5-a2c5-80c80a698837 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:24.528709 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f5ef6-074c-46f5-a2c5-80c80a698837 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:24.529145 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f5ef6-074c-46f5-a2c5-80c80a698837 None admin] PATH_INFO: `/v3/projects/c18388255f054822b773ec88708292da` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:24.531417 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f5ef6-074c-46f5-a2c5-80c80a698837 None admin] RBAC: Authorizing `identity:get_project(project_id=c18388255f054822b773ec88708292da)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:24.534566 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f5ef6-074c-46f5-a2c5-80c80a698837 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:24.541857 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 173/356] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:53:24 2022] GET /identity/v3/projects/c18388255f054822b773ec88708292da => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:24.541857 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937541cd-d1f6-4980-9aeb-0720f831e2d6 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:24.541857 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937541cd-d1f6-4980-9aeb-0720f831e2d6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:24.541857 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937541cd-d1f6-4980-9aeb-0720f831e2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:24.564728 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-937541cd-d1f6-4980-9aeb-0720f831e2d6 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:24.566230 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-937541cd-d1f6-4980-9aeb-0720f831e2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:24.641392 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56e8a1-eda8-4b17-8df8-a537f56aff23 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:24.688383 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 174/357] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:24.688631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937541cd-d1f6-4980-9aeb-0720f831e2d6 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:24.699614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c6a73-bb6f-45c4-9427-8d54333f4c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:24.700982 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c6a73-bb6f-45c4-9427-8d54333f4c84 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:24.701276 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c6a73-bb6f-45c4-9427-8d54333f4c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:24.701539 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c6a73-bb6f-45c4-9427-8d54333f4c84 None admin] PATH_INFO: `/v3/groups/6074835adfc045b496c12ccdf45c7672` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:24.710930 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1c6a73-bb6f-45c4-9427-8d54333f4c84 None admin] RBAC: Authorizing `identity:get_group(group_id=6074835adfc045b496c12ccdf45c7672)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:24.712842 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1c6a73-bb6f-45c4-9427-8d54333f4c84 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:24.715179 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 184/358] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Aug 1 13:53:24 2022] GET /identity/v3/groups/6074835adfc045b496c12ccdf45c7672 => generated 226 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:24.721512 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c6a73-bb6f-45c4-9427-8d54333f4c84 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:24.738214 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:24.815439 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:24.851709 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937541cd-d1f6-4980-9aeb-0720f831e2d6 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:24.878910 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c6a73-bb6f-45c4-9427-8d54333f4c84 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:24.910633 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f6fdb-aa6d-48c8-8c77-624d297acb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:24.911594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f6fdb-aa6d-48c8-8c77-624d297acb77 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:24.912035 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f6fdb-aa6d-48c8-8c77-624d297acb77 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:24.912505 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f6fdb-aa6d-48c8-8c77-624d297acb77 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:24.913750 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f6fdb-aa6d-48c8-8c77-624d297acb77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:24.914182 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f6fdb-aa6d-48c8-8c77-624d297acb77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:24.915757 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f6fdb-aa6d-48c8-8c77-624d297acb77 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:24.927754 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 175/359] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:53:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:24.929274 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f6fdb-aa6d-48c8-8c77-624d297acb77 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:24.941740 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f3060e-b678-40ce-8d18-1de2e418911d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:24.942400 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f3060e-b678-40ce-8d18-1de2e418911d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:24.942681 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f3060e-b678-40ce-8d18-1de2e418911d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:24.942957 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f3060e-b678-40ce-8d18-1de2e418911d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:25.044263 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f3060e-b678-40ce-8d18-1de2e418911d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:25.045934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f3060e-b678-40ce-8d18-1de2e418911d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:25.059060 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:25.085337 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f6fdb-aa6d-48c8-8c77-624d297acb77 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:25.129862 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31aca9e-48fe-4bfc-b2e2-dcd1ebb55670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:25.130624 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31aca9e-48fe-4bfc-b2e2-dcd1ebb55670 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:25.131069 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31aca9e-48fe-4bfc-b2e2-dcd1ebb55670 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:25.131424 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31aca9e-48fe-4bfc-b2e2-dcd1ebb55670 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:25.141404 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:25.146293 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31aca9e-48fe-4bfc-b2e2-dcd1ebb55670 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:25.152062 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-d31aca9e-48fe-4bfc-b2e2-dcd1ebb55670 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:53:25.152062 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:25.152062 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:25.152062 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:25.152062 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:25.152062 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:25.152062 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:25.152062 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:25.152062 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:25.152062 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:25.152062 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:25.152062 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:25.152062 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:25.152062 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:25.152062 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:25.152062 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:25.154181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:25.154181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:25.154181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:25.154181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:25.154181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:25.154181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:25.154181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:25.154181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:25.154181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:25.154181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:25.154181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:25.154181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:25.154181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:25.154181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:25.154181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:25.154181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:25.155974 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:25.155974 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:25.155974 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:25.155974 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:25.155974 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:25.155974 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:25.155974 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:25.155974 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:25.155974 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:25.155974 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:25.155974 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:53:25.155974 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:53:25.155974 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 176/360] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 13:53:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:25.157336 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31aca9e-48fe-4bfc-b2e2-dcd1ebb55670 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:25.183558 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 185/361] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:53:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:53:25.185521 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f3060e-b678-40ce-8d18-1de2e418911d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:25.281472 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31aca9e-48fe-4bfc-b2e2-dcd1ebb55670 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:25.306917 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f3060e-b678-40ce-8d18-1de2e418911d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:25.318003 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:25.320977 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db5cff-e8fd-4eb6-845a-a17cf83512b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:25.321978 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db5cff-e8fd-4eb6-845a-a17cf83512b2 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:25.322330 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db5cff-e8fd-4eb6-845a-a17cf83512b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:25.322663 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db5cff-e8fd-4eb6-845a-a17cf83512b2 None admin] PATH_INFO: `/v3/groups/6074835adfc045b496c12ccdf45c7672` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:25.324057 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98db5cff-e8fd-4eb6-845a-a17cf83512b2 None admin] RBAC: Authorizing `identity:get_group(group_id=6074835adfc045b496c12ccdf45c7672)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:25.325951 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98db5cff-e8fd-4eb6-845a-a17cf83512b2 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:25.327997 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 177/362] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Aug 1 13:53:25 2022] GET /identity/v3/groups/6074835adfc045b496c12ccdf45c7672 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:25.328958 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db5cff-e8fd-4eb6-845a-a17cf83512b2 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:25.348020 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d426f-3038-456e-87d2-dfceef78164f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:25.348600 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d426f-3038-456e-87d2-dfceef78164f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:25.348885 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d426f-3038-456e-87d2-dfceef78164f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:25.349154 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d426f-3038-456e-87d2-dfceef78164f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:25.349962 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1d426f-3038-456e-87d2-dfceef78164f None admin] RBAC: Adding query filter params (group.id=6074835adfc045b496c12ccdf45c7672, role.id=75aff8cb4cf748a2adbd6c3f141fe8ac, scope.project.id=c18388255f054822b773ec88708292da) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:25.350264 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1d426f-3038-456e-87d2-dfceef78164f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:25.351716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1d426f-3038-456e-87d2-dfceef78164f None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:25.358223 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 186/363] 10.222.0.45 () {56 vars in 1376 bytes} [Mon Aug 1 13:53:25 2022] GET /identity/v3/role_assignments?group.id=6074835adfc045b496c12ccdf45c7672&scope.project.id=c18388255f054822b773ec88708292da&role.id=75aff8cb4cf748a2adbd6c3f141fe8ac => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:25.361263 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d426f-3038-456e-87d2-dfceef78164f None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:25.415880 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:25.466728 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db5cff-e8fd-4eb6-845a-a17cf83512b2 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:25.528810 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8422a91-a442-42a4-8b3e-0c65435645fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:25.529595 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d426f-3038-456e-87d2-dfceef78164f None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:25.529812 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8422a91-a442-42a4-8b3e-0c65435645fc None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:25.530246 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8422a91-a442-42a4-8b3e-0c65435645fc None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:25.530701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8422a91-a442-42a4-8b3e-0c65435645fc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:25.554503 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8422a91-a442-42a4-8b3e-0c65435645fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:25.565785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-f8422a91-a442-42a4-8b3e-0c65435645fc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:53:25.565785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:25.565785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:25.565785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:25.565785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:25.565785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:25.565785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:25.565785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:25.565785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:25.565785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:25.565785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:25.565785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:25.565785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:25.565785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:25.565785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:25.565785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:25.566790 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:25.566790 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:25.566790 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:25.566790 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:25.566790 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:25.566790 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:25.566790 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:25.566790 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:25.566790 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:25.566790 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:25.566790 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:25.566790 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:25.566790 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:25.566790 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:25.566790 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:25.566790 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:25.567634 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:25.567634 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:25.567634 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:25.567634 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:25.567634 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:25.567634 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:25.567634 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:25.567634 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:25.567634 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:25.567634 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:25.567634 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:53:25.567634 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:53:25.568313 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 178/364] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:53:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:25.569324 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8422a91-a442-42a4-8b3e-0c65435645fc None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:25.588479 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22beb045-66fe-4313-99c4-c53f1f54a1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:25.589296 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22beb045-66fe-4313-99c4-c53f1f54a1f1 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:25.589417 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22beb045-66fe-4313-99c4-c53f1f54a1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:25.589674 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22beb045-66fe-4313-99c4-c53f1f54a1f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:25.649606 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:25.702528 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22beb045-66fe-4313-99c4-c53f1f54a1f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:25.706241 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22beb045-66fe-4313-99c4-c53f1f54a1f1 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:25.713760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8422a91-a442-42a4-8b3e-0c65435645fc None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:25.754819 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:25.766084 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cadd5fb-3ef3-45d5-9277-ac36b568a9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:25.767032 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadd5fb-3ef3-45d5-9277-ac36b568a9d1 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:25.767435 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadd5fb-3ef3-45d5-9277-ac36b568a9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:25.767887 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadd5fb-3ef3-45d5-9277-ac36b568a9d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:25.769954 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cadd5fb-3ef3-45d5-9277-ac36b568a9d1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:25.769954 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cadd5fb-3ef3-45d5-9277-ac36b568a9d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:25.771790 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cadd5fb-3ef3-45d5-9277-ac36b568a9d1 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:25.783921 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 179/365] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:53:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:25.785045 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cadd5fb-3ef3-45d5-9277-ac36b568a9d1 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:25.866559 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 187/366] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:53:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:53:25.868134 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22beb045-66fe-4313-99c4-c53f1f54a1f1 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:25.931522 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cadd5fb-3ef3-45d5-9277-ac36b568a9d1 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:25.986879 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21a361-dde5-427e-837a-c811a8654dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:25.987940 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21a361-dde5-427e-837a-c811a8654dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:25.988471 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21a361-dde5-427e-837a-c811a8654dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:25.988913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21a361-dde5-427e-837a-c811a8654dd5 None admin] PATH_INFO: `/v3/projects/7489ab03e4a8475cab57be63c14ec0c0` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:25.990632 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c21a361-dde5-427e-837a-c811a8654dd5 None admin] RBAC: Authorizing `identity:get_project(project_id=7489ab03e4a8475cab57be63c14ec0c0)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:25.993380 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c21a361-dde5-427e-837a-c811a8654dd5 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:25.998975 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:26.000723 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 180/367] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:53:25 2022] GET /identity/v3/projects/7489ab03e4a8475cab57be63c14ec0c0 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:26.002339 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21a361-dde5-427e-837a-c811a8654dd5 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:26.038842 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22beb045-66fe-4313-99c4-c53f1f54a1f1 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:26.105683 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c882de-c0cb-46dd-9e8e-782c1734a724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:26.106777 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c882de-c0cb-46dd-9e8e-782c1734a724 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:26.107032 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:26.107655 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c882de-c0cb-46dd-9e8e-782c1734a724 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:26.107791 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c882de-c0cb-46dd-9e8e-782c1734a724 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:26.109311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c882de-c0cb-46dd-9e8e-782c1734a724 None admin] RBAC: Adding query filter params (group.id=6074835adfc045b496c12ccdf45c7672, role.id=60efb70fa7354ebfb04aa5e9bdad2a78, scope.project.id=c18388255f054822b773ec88708292da) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:26.109311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c882de-c0cb-46dd-9e8e-782c1734a724 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:26.111355 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c882de-c0cb-46dd-9e8e-782c1734a724 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:26.128343 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 188/368] 10.222.0.45 () {56 vars in 1376 bytes} [Mon Aug 1 13:53:25 2022] GET /identity/v3/role_assignments?group.id=6074835adfc045b496c12ccdf45c7672&scope.project.id=c18388255f054822b773ec88708292da&role.id=60efb70fa7354ebfb04aa5e9bdad2a78 => generated 260 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:26.128343 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c882de-c0cb-46dd-9e8e-782c1734a724 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:26.203953 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21a361-dde5-427e-837a-c811a8654dd5 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:26.272266 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4e863-683f-45a1-9702-b0339c15ba19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:26.273511 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4e863-683f-45a1-9702-b0339c15ba19 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:26.274082 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4e863-683f-45a1-9702-b0339c15ba19 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:26.274657 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4e863-683f-45a1-9702-b0339c15ba19 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:26.276344 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4e863-683f-45a1-9702-b0339c15ba19 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:26.276964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4e863-683f-45a1-9702-b0339c15ba19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:26.279736 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4e863-683f-45a1-9702-b0339c15ba19 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:26.298350 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 181/369] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 13:53:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:26.300153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4e863-683f-45a1-9702-b0339c15ba19 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:26.362616 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c882de-c0cb-46dd-9e8e-782c1734a724 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:26.432681 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ff155-f545-4a88-a64c-22f700671ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:26.433908 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ff155-f545-4a88-a64c-22f700671ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:26.434528 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ff155-f545-4a88-a64c-22f700671ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:26.435097 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ff155-f545-4a88-a64c-22f700671ae6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:26.442303 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:26.501554 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4e863-683f-45a1-9702-b0339c15ba19 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:26.557311 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:26.574058 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd1f0c-08a4-4a7e-a2d0-d22c62b927cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:26.574988 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd1f0c-08a4-4a7e-a2d0-d22c62b927cd None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:26.575298 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd1f0c-08a4-4a7e-a2d0-d22c62b927cd None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:26.575698 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd1f0c-08a4-4a7e-a2d0-d22c62b927cd None admin] PATH_INFO: `/v3/projects/2a12613c4ac44ad788edb767368b347b` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:26.577461 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd1f0c-08a4-4a7e-a2d0-d22c62b927cd None admin] RBAC: Authorizing `identity:get_project(project_id=2a12613c4ac44ad788edb767368b347b)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:26.579680 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd1f0c-08a4-4a7e-a2d0-d22c62b927cd None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:26.581301 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ff155-f545-4a88-a64c-22f700671ae6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:26.582474 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 182/370] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:53:26 2022] GET /identity/v3/projects/2a12613c4ac44ad788edb767368b347b => generated 344 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:26.583676 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd1f0c-08a4-4a7e-a2d0-d22c62b927cd None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:26.589480 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ff155-f545-4a88-a64c-22f700671ae6 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:26.784543 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd1f0c-08a4-4a7e-a2d0-d22c62b927cd None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:26.812556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 189/371] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:53:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 687 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:53:26.814412 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ff155-f545-4a88-a64c-22f700671ae6 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:26.851313 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6954ccbd-58b2-4b45-b9dc-e2b525e148fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:26.852493 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954ccbd-58b2-4b45-b9dc-e2b525e148fe None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:26.852991 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954ccbd-58b2-4b45-b9dc-e2b525e148fe None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:26.853553 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954ccbd-58b2-4b45-b9dc-e2b525e148fe None admin] PATH_INFO: `/v3/groups/6074835adfc045b496c12ccdf45c7672` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:26.855714 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6954ccbd-58b2-4b45-b9dc-e2b525e148fe None admin] RBAC: Authorizing `identity:get_group(group_id=6074835adfc045b496c12ccdf45c7672)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:26.858822 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6954ccbd-58b2-4b45-b9dc-e2b525e148fe None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:26.862037 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 183/372] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Aug 1 13:53:26 2022] GET /identity/v3/groups/6074835adfc045b496c12ccdf45c7672 => generated 226 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:26.863490 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6954ccbd-58b2-4b45-b9dc-e2b525e148fe None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:26.889780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:26.978301 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:27.021935 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ff155-f545-4a88-a64c-22f700671ae6 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:27.059281 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6954ccbd-58b2-4b45-b9dc-e2b525e148fe None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:27.091994 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b003dcb-20c6-488b-a564-f4f3d12a1313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:27.093194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b003dcb-20c6-488b-a564-f4f3d12a1313 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:27.093759 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b003dcb-20c6-488b-a564-f4f3d12a1313 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:27.094232 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b003dcb-20c6-488b-a564-f4f3d12a1313 None admin] PATH_INFO: `/v3/users/66e04eaffcdb442896ecb3a531bf2c65` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:27.096777 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b003dcb-20c6-488b-a564-f4f3d12a1313 None admin] RBAC: Authorizing `identity:get_user(user_id=66e04eaffcdb442896ecb3a531bf2c65)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:27.099581 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b003dcb-20c6-488b-a564-f4f3d12a1313 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:27.103375 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 190/373] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:53:26 2022] GET /identity/v3/users/66e04eaffcdb442896ecb3a531bf2c65 => generated 293 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:27.105099 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b003dcb-20c6-488b-a564-f4f3d12a1313 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:27.137181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a6ccd-a632-4146-9b64-87b4815aaf55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:27.138200 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a6ccd-a632-4146-9b64-87b4815aaf55 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:27.138715 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a6ccd-a632-4146-9b64-87b4815aaf55 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:27.139210 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a6ccd-a632-4146-9b64-87b4815aaf55 None admin] PATH_INFO: `/v3/groups/a8b1953358064ba4b85be4020d28d938` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:27.151545 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a6ccd-a632-4146-9b64-87b4815aaf55 None admin] RBAC: Authorizing `identity:get_group(group_id=a8b1953358064ba4b85be4020d28d938)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:27.154848 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a6ccd-a632-4146-9b64-87b4815aaf55 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:27.158126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 184/374] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Aug 1 13:53:26 2022] GET /identity/v3/groups/a8b1953358064ba4b85be4020d28d938 => generated 229 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:27.159630 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a6ccd-a632-4146-9b64-87b4815aaf55 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:27.313187 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b003dcb-20c6-488b-a564-f4f3d12a1313 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:27.318020 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:27.322155 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a6ccd-a632-4146-9b64-87b4815aaf55 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:27.347401 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:27.367823 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632cda8f-34a4-4367-9610-27c6392d4b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:27.368504 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cda8f-34a4-4367-9610-27c6392d4b33 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:27.368779 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cda8f-34a4-4367-9610-27c6392d4b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:27.369162 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cda8f-34a4-4367-9610-27c6392d4b33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:27.369985 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632cda8f-34a4-4367-9610-27c6392d4b33 None admin] RBAC: Adding query filter params (group.id=6074835adfc045b496c12ccdf45c7672, role.id=75aff8cb4cf748a2adbd6c3f141fe8ac, scope.project.id=7489ab03e4a8475cab57be63c14ec0c0) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:27.370297 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632cda8f-34a4-4367-9610-27c6392d4b33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:27.371913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632cda8f-34a4-4367-9610-27c6392d4b33 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:27.379334 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 185/375] 10.222.0.45 () {56 vars in 1376 bytes} [Mon Aug 1 13:53:27 2022] GET /identity/v3/role_assignments?group.id=6074835adfc045b496c12ccdf45c7672&scope.project.id=7489ab03e4a8475cab57be63c14ec0c0&role.id=75aff8cb4cf748a2adbd6c3f141fe8ac => generated 260 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:27.380238 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632cda8f-34a4-4367-9610-27c6392d4b33 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:27.397537 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c217b46d-9f6b-45a7-8861-ebb291c88522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:27.398741 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217b46d-9f6b-45a7-8861-ebb291c88522 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:27.399228 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217b46d-9f6b-45a7-8861-ebb291c88522 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:27.399859 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217b46d-9f6b-45a7-8861-ebb291c88522 None admin] PATH_INFO: `/v3/projects/7489ab03e4a8475cab57be63c14ec0c0` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:27.406087 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c217b46d-9f6b-45a7-8861-ebb291c88522 None admin] RBAC: Authorizing `identity:get_project(project_id=7489ab03e4a8475cab57be63c14ec0c0)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:27.409132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c217b46d-9f6b-45a7-8861-ebb291c88522 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:27.412942 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 191/376] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:53:27 2022] GET /identity/v3/projects/7489ab03e4a8475cab57be63c14ec0c0 => generated 302 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:27.421423 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c217b46d-9f6b-45a7-8861-ebb291c88522 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:27.571272 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632cda8f-34a4-4367-9610-27c6392d4b33 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:27.701257 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a434d2a8-1830-4ac9-b2a5-a8364640d9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:27.704643 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434d2a8-1830-4ac9-b2a5-a8364640d9c0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:27.705140 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434d2a8-1830-4ac9-b2a5-a8364640d9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:27.705616 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434d2a8-1830-4ac9-b2a5-a8364640d9c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:27.708119 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a434d2a8-1830-4ac9-b2a5-a8364640d9c0 None admin] RBAC: Adding query filter params (role.id=9db7d52dd9514261a5a52591fbf1feef, user.id=66e04eaffcdb442896ecb3a531bf2c65, scope.system=all) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:27.708723 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a434d2a8-1830-4ac9-b2a5-a8364640d9c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:27.711485 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a434d2a8-1830-4ac9-b2a5-a8364640d9c0 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:27.751611 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c217b46d-9f6b-45a7-8861-ebb291c88522 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:27.757087 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 186/377] 10.222.0.45 () {56 vars in 1308 bytes} [Mon Aug 1 13:53:27 2022] GET /identity/v3/role_assignments?user.id=66e04eaffcdb442896ecb3a531bf2c65&scope.system=all&role.id=9db7d52dd9514261a5a52591fbf1feef => generated 510 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:27.758637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a434d2a8-1830-4ac9-b2a5-a8364640d9c0 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:27.825369 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b0a9a4-7e2a-4a86-8cd9-e5d174d147ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:27.826381 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0a9a4-7e2a-4a86-8cd9-e5d174d147ab None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:27.826878 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0a9a4-7e2a-4a86-8cd9-e5d174d147ab None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:27.827430 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0a9a4-7e2a-4a86-8cd9-e5d174d147ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:27.829409 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b0a9a4-7e2a-4a86-8cd9-e5d174d147ab None admin] RBAC: Adding query filter params (group.id=a8b1953358064ba4b85be4020d28d938, role.id=3e1a9ff016c840c5b51ef7e3da9900ac, scope.project.id=2a12613c4ac44ad788edb767368b347b) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:27.829842 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b0a9a4-7e2a-4a86-8cd9-e5d174d147ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:27.832490 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b0a9a4-7e2a-4a86-8cd9-e5d174d147ab None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:27.847476 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 192/378] 10.222.0.45 () {56 vars in 1376 bytes} [Mon Aug 1 13:53:27 2022] GET /identity/v3/role_assignments?group.id=a8b1953358064ba4b85be4020d28d938&scope.project.id=2a12613c4ac44ad788edb767368b347b&role.id=3e1a9ff016c840c5b51ef7e3da9900ac => generated 260 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:27.895546 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:28.045475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a434d2a8-1830-4ac9-b2a5-a8364640d9c0 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:28.121725 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c978b64-d617-405c-b110-0f5ef3e845f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:28.122799 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c978b64-d617-405c-b110-0f5ef3e845f3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:28.123222 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c978b64-d617-405c-b110-0f5ef3e845f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:28.123677 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c978b64-d617-405c-b110-0f5ef3e845f3 None admin] PATH_INFO: `/v3/groups/6074835adfc045b496c12ccdf45c7672` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:28.127172 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c978b64-d617-405c-b110-0f5ef3e845f3 None admin] RBAC: Authorizing `identity:get_group(group_id=6074835adfc045b496c12ccdf45c7672)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:28.129752 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c978b64-d617-405c-b110-0f5ef3e845f3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:28.132971 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 187/379] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Aug 1 13:53:27 2022] GET /identity/v3/groups/6074835adfc045b496c12ccdf45c7672 => generated 226 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:28.140333 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b0a9a4-7e2a-4a86-8cd9-e5d174d147ab None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:28.218188 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:28.336401 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b0a9a4-7e2a-4a86-8cd9-e5d174d147ab None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:28.395564 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87af18a-5013-438e-85b5-ba3aab232ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:28.396553 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87af18a-5013-438e-85b5-ba3aab232ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:28.397058 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87af18a-5013-438e-85b5-ba3aab232ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:28.397524 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87af18a-5013-438e-85b5-ba3aab232ae3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:28.398896 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87af18a-5013-438e-85b5-ba3aab232ae3 None admin] RBAC: Adding query filter params (group.id=6074835adfc045b496c12ccdf45c7672, role.id=60efb70fa7354ebfb04aa5e9bdad2a78, scope.project.id=7489ab03e4a8475cab57be63c14ec0c0) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:28.399514 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87af18a-5013-438e-85b5-ba3aab232ae3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:28.402004 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87af18a-5013-438e-85b5-ba3aab232ae3 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:28.412566 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 193/380] 10.222.0.45 () {56 vars in 1376 bytes} [Mon Aug 1 13:53:28 2022] GET /identity/v3/role_assignments?group.id=6074835adfc045b496c12ccdf45c7672&scope.project.id=7489ab03e4a8475cab57be63c14ec0c0&role.id=60efb70fa7354ebfb04aa5e9bdad2a78 => generated 260 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:28.725423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c1579-ba2b-41fa-bae1-738d06c6b62e None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:28.726368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c1579-ba2b-41fa-bae1-738d06c6b62e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:28.726368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c1579-ba2b-41fa-bae1-738d06c6b62e None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:28.727669 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 188/381] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:28.735394 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30504ae-3042-408a-84e8-de19b297f5d8 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:28.735797 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30504ae-3042-408a-84e8-de19b297f5d8 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:28.736009 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30504ae-3042-408a-84e8-de19b297f5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:28.764567 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-c30504ae-3042-408a-84e8-de19b297f5d8 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:28.766816 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c30504ae-3042-408a-84e8-de19b297f5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:28.778975 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a06f52-c165-451c-b8ad-6d4ab4570ad6 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:28.779634 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a06f52-c165-451c-b8ad-6d4ab4570ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:28.780124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a06f52-c165-451c-b8ad-6d4ab4570ad6 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:28.782839 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 189/382] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:28.796240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46521bc9-9f31-479b-8354-d39e6f5549e0 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:28.797142 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46521bc9-9f31-479b-8354-d39e6f5549e0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:28.797781 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46521bc9-9f31-479b-8354-d39e6f5549e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:28.829939 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-46521bc9-9f31-479b-8354-d39e6f5549e0 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:28.831627 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-46521bc9-9f31-479b-8354-d39e6f5549e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:28.885012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 194/383] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:53:28.902780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30504ae-3042-408a-84e8-de19b297f5d8 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:28.940849 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:29.028156 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30504ae-3042-408a-84e8-de19b297f5d8 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:29.070789 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e36988a-cf2c-40a1-a34a-ebb503815f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:29.076305 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36988a-cf2c-40a1-a34a-ebb503815f1b None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:29.076305 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36988a-cf2c-40a1-a34a-ebb503815f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:29.076305 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36988a-cf2c-40a1-a34a-ebb503815f1b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:29.089982 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e36988a-cf2c-40a1-a34a-ebb503815f1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:29.091331 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 190/384] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:53:29.098094 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-9e36988a-cf2c-40a1-a34a-ebb503815f1b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:53:29.098094 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:29.098094 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:29.098094 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:29.098094 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:29.098094 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:29.098094 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:29.098094 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:29.098094 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:29.098094 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:29.098094 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:29.098094 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:29.098094 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:29.098094 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:29.098094 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:29.098094 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:29.099055 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:29.099055 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:29.099055 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:29.099055 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:29.099055 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:29.099055 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:29.099055 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:29.099055 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:29.099055 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:29.099055 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:29.099055 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:29.099055 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:29.099055 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:29.099055 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:29.099055 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:29.099055 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:29.099877 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:29.099877 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:29.099877 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:29.099877 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:29.099877 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:29.099877 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:29.099877 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:29.099877 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:29.099877 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:29.099877 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:29.099877 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:53:29.099877 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:29.102054 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 195/385] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:53:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:29.109916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46521bc9-9f31-479b-8354-d39e6f5549e0 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:29.165522 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:29.171904 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e36988a-cf2c-40a1-a34a-ebb503815f1b None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:29.335869 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:29.345032 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46521bc9-9f31-479b-8354-d39e6f5549e0 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:29.345301 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e36988a-cf2c-40a1-a34a-ebb503815f1b None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:29.388644 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d69ebba-adb2-4230-83fa-82d2d3a93c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:29.389606 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69ebba-adb2-4230-83fa-82d2d3a93c9f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:29.390147 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69ebba-adb2-4230-83fa-82d2d3a93c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:29.390555 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69ebba-adb2-4230-83fa-82d2d3a93c9f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:29.407105 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d69ebba-adb2-4230-83fa-82d2d3a93c9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:29.413821 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-6d69ebba-adb2-4230-83fa-82d2d3a93c9f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:53:29.413821 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:29.413821 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:29.413821 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:29.413821 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:29.413821 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:29.413821 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:29.413821 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:29.413821 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:29.413821 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:29.413821 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:29.413821 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:29.413821 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:29.413821 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:29.413821 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:29.413821 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:29.414936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:29.414936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:29.414936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:29.414936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:29.414936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:29.414936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:29.414936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:29.414936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:29.414936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:29.414936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:29.414936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:29.414936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:29.414936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:29.414936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:29.414936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:29.414936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:29.415860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:29.415860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:29.415860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:29.415860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:29.415860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:29.415860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:29.415860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:29.415860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:29.415860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:29.415860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:29.415860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:53:29.415860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:29.418844 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 196/386] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 13:53:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:29.427300 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d69ebba-adb2-4230-83fa-82d2d3a93c9f None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:29.473628 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d51069-2151-402e-9e39-cd6d84bbdd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:29.473999 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d51069-2151-402e-9e39-cd6d84bbdd47 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:29.474488 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d51069-2151-402e-9e39-cd6d84bbdd47 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:29.474937 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d51069-2151-402e-9e39-cd6d84bbdd47 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:29.476190 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d51069-2151-402e-9e39-cd6d84bbdd47 None admin] RBAC: Adding query filter params (name=member) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:29.476750 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d51069-2151-402e-9e39-cd6d84bbdd47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:29.479414 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d51069-2151-402e-9e39-cd6d84bbdd47 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:29.508316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 191/387] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 13:53:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:29.544265 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d51069-2151-402e-9e39-cd6d84bbdd47 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:29.600206 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d69ebba-adb2-4230-83fa-82d2d3a93c9f None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:29.680903 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979436e-8ec7-4a03-9dda-11fb130a2654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:29.682461 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979436e-8ec7-4a03-9dda-11fb130a2654 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:29.682965 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979436e-8ec7-4a03-9dda-11fb130a2654 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:29.683606 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979436e-8ec7-4a03-9dda-11fb130a2654 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:29.685072 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c979436e-8ec7-4a03-9dda-11fb130a2654 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:29.685632 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c979436e-8ec7-4a03-9dda-11fb130a2654 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:29.689658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c979436e-8ec7-4a03-9dda-11fb130a2654 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:29.709850 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 197/388] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:53:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:53:29.720840 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979436e-8ec7-4a03-9dda-11fb130a2654 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:29.765597 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d51069-2151-402e-9e39-cd6d84bbdd47 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:29.767566 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:29.772025 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:29.824935 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94546848-0ff2-4c0d-847f-0d5b498c527c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:29.826135 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94546848-0ff2-4c0d-847f-0d5b498c527c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:29.826614 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94546848-0ff2-4c0d-847f-0d5b498c527c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:29.827099 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94546848-0ff2-4c0d-847f-0d5b498c527c None admin] PATH_INFO: `/v3/groups/6074835adfc045b496c12ccdf45c7672` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:29.829367 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94546848-0ff2-4c0d-847f-0d5b498c527c None admin] RBAC: Authorizing `identity:get_group(group_id=6074835adfc045b496c12ccdf45c7672)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:29.832014 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94546848-0ff2-4c0d-847f-0d5b498c527c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:29.835819 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 192/389] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Aug 1 13:53:29 2022] GET /identity/v3/groups/6074835adfc045b496c12ccdf45c7672 => generated 226 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:29.859077 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94546848-0ff2-4c0d-847f-0d5b498c527c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:29.949888 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979436e-8ec7-4a03-9dda-11fb130a2654 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:30.019687 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c341fd-1f2d-4ca3-bbe1-7f88c4753356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:30.020850 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c341fd-1f2d-4ca3-bbe1-7f88c4753356 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:30.021473 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c341fd-1f2d-4ca3-bbe1-7f88c4753356 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:30.022095 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c341fd-1f2d-4ca3-bbe1-7f88c4753356 None admin] PATH_INFO: `/v3/groups/6074835adfc045b496c12ccdf45c7672` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:30.024189 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c341fd-1f2d-4ca3-bbe1-7f88c4753356 None admin] RBAC: Authorizing `identity:get_group(group_id=6074835adfc045b496c12ccdf45c7672)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:30.027291 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c341fd-1f2d-4ca3-bbe1-7f88c4753356 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:30.028183 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94546848-0ff2-4c0d-847f-0d5b498c527c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:30.030579 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 198/390] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Aug 1 13:53:29 2022] GET /identity/v3/groups/6074835adfc045b496c12ccdf45c7672 => generated 226 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:30.040098 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c341fd-1f2d-4ca3-bbe1-7f88c4753356 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:30.068751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1498180-cd73-4c99-abe6-79e2299fb891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:30.069763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1498180-cd73-4c99-abe6-79e2299fb891 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:30.070229 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1498180-cd73-4c99-abe6-79e2299fb891 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:30.070705 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1498180-cd73-4c99-abe6-79e2299fb891 None admin] PATH_INFO: `/v3/projects/c18388255f054822b773ec88708292da` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:30.072631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1498180-cd73-4c99-abe6-79e2299fb891 None admin] RBAC: Authorizing `identity:get_project(project_id=c18388255f054822b773ec88708292da)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:30.075245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1498180-cd73-4c99-abe6-79e2299fb891 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:30.078869 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 193/391] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:53:29 2022] GET /identity/v3/projects/c18388255f054822b773ec88708292da => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:30.097872 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1498180-cd73-4c99-abe6-79e2299fb891 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:30.173824 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:30.241815 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1498180-cd73-4c99-abe6-79e2299fb891 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:30.297189 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:30.303931 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cbcbb3-8838-40fc-ad21-77671167b7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:30.305833 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbcbb3-8838-40fc-ad21-77671167b7d7 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:30.306536 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbcbb3-8838-40fc-ad21-77671167b7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:30.307429 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbcbb3-8838-40fc-ad21-77671167b7d7 None admin] PATH_INFO: `/v3/projects/c18388255f054822b773ec88708292da/groups/6074835adfc045b496c12ccdf45c7672/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:30.318901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c341fd-1f2d-4ca3-bbe1-7f88c4753356 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:30.319629 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cbcbb3-8838-40fc-ad21-77671167b7d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=c18388255f054822b773ec88708292da, group_id=6074835adfc045b496c12ccdf45c7672, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:30.322428 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cbcbb3-8838-40fc-ad21-77671167b7d7 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:30.370700 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81856d0-032d-4981-a96d-dcc6cc2bb19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:30.371929 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81856d0-032d-4981-a96d-dcc6cc2bb19a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:30.372446 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81856d0-032d-4981-a96d-dcc6cc2bb19a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:30.373094 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81856d0-032d-4981-a96d-dcc6cc2bb19a None admin] PATH_INFO: `/v3/projects/c18388255f054822b773ec88708292da` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:30.375625 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81856d0-032d-4981-a96d-dcc6cc2bb19a None admin] RBAC: Authorizing `identity:get_project(project_id=c18388255f054822b773ec88708292da)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:30.378431 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81856d0-032d-4981-a96d-dcc6cc2bb19a None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:30.381183 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 194/392] 10.222.0.45 () {58 vars in 1442 bytes} [Mon Aug 1 13:53:30 2022] PUT /identity/v3/projects/c18388255f054822b773ec88708292da/groups/6074835adfc045b496c12ccdf45c7672/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:53:30.381935 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 199/393] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:53:30 2022] GET /identity/v3/projects/c18388255f054822b773ec88708292da => generated 298 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:30.391424 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cbcbb3-8838-40fc-ad21-77671167b7d7 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:30.604927 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:30.610352 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cbcbb3-8838-40fc-ad21-77671167b7d7 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:30.683497 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7434aa-9eb2-4011-8cd1-07563246aeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:30.684770 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7434aa-9eb2-4011-8cd1-07563246aeda None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:30.685254 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7434aa-9eb2-4011-8cd1-07563246aeda None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:30.685734 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7434aa-9eb2-4011-8cd1-07563246aeda None admin] PATH_INFO: `/v3/projects/c18388255f054822b773ec88708292da/groups/6074835adfc045b496c12ccdf45c7672/roles/60efb70fa7354ebfb04aa5e9bdad2a78` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:30.689984 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7434aa-9eb2-4011-8cd1-07563246aeda None admin] RBAC: Authorizing `identity:create_grant(project_id=c18388255f054822b773ec88708292da, group_id=6074835adfc045b496c12ccdf45c7672, role_id=60efb70fa7354ebfb04aa5e9bdad2a78)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:30.692930 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7434aa-9eb2-4011-8cd1-07563246aeda None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:30.719695 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 195/394] 10.222.0.45 () {58 vars in 1442 bytes} [Mon Aug 1 13:53:30 2022] PUT /identity/v3/projects/c18388255f054822b773ec88708292da/groups/6074835adfc045b496c12ccdf45c7672/roles/60efb70fa7354ebfb04aa5e9bdad2a78 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:53:31.379213 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30da54-b7b9-463c-aa6c-922acc321d22 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:31.379992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30da54-b7b9-463c-aa6c-922acc321d22 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:31.380426 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30da54-b7b9-463c-aa6c-922acc321d22 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:31.382983 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 200/395] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:31.392078 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333cfc8-6ddf-47b1-b2eb-7f9c179e4819 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:31.392623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333cfc8-6ddf-47b1-b2eb-7f9c179e4819 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:31.393384 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333cfc8-6ddf-47b1-b2eb-7f9c179e4819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:31.435575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b333cfc8-6ddf-47b1-b2eb-7f9c179e4819 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:31.437819 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b333cfc8-6ddf-47b1-b2eb-7f9c179e4819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:31.449974 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e195c-11f5-4047-b052-22286ced0dda None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:31.450353 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e195c-11f5-4047-b052-22286ced0dda None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:31.450528 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e195c-11f5-4047-b052-22286ced0dda None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:31.456431 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 201/396] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:31.459352 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93332e1f-919a-4520-9a46-845dc2df876a None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:31.459668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93332e1f-919a-4520-9a46-845dc2df876a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:31.459942 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93332e1f-919a-4520-9a46-845dc2df876a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:31.493435 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-93332e1f-919a-4520-9a46-845dc2df876a None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:31.495423 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-93332e1f-919a-4520-9a46-845dc2df876a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:31.666548 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 196/397] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:53:31.669311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 202/398] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:53:31.691541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333cfc8-6ddf-47b1-b2eb-7f9c179e4819 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:31.706714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93332e1f-919a-4520-9a46-845dc2df876a None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:31.887633 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:31.899542 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:31.930339 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93332e1f-919a-4520-9a46-845dc2df876a None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:31.935980 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333cfc8-6ddf-47b1-b2eb-7f9c179e4819 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:31.977240 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6abc1d-6cf7-4609-9a19-204ad8921b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:31.978021 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6abc1d-6cf7-4609-9a19-204ad8921b63 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:31.978371 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6abc1d-6cf7-4609-9a19-204ad8921b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:31.978710 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6abc1d-6cf7-4609-9a19-204ad8921b63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:31.992356 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6abc1d-6cf7-4609-9a19-204ad8921b63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:31.998037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-bd6abc1d-6cf7-4609-9a19-204ad8921b63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:53:31.998037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:31.998037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:31.998037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:31.998037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:31.998037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:31.998037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:31.998037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:31.998037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:31.998037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:31.998037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:31.998037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:31.998037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:31.998037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:31.998037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:31.998037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:32.000177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:32.000177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:32.000177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:32.000177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:32.000177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:32.000177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:32.000177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:32.000177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:32.000177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:32.000177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:32.000177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:32.000177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:32.000177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:32.000177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:32.000177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:32.000177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:32.002019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:32.002019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:32.002019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:32.002019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:32.002019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:32.002019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:32.002019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:32.002019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:32.002019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:32.002019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:32.002019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:53:32.002019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:32.002019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 203/399] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:53:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:32.007194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f662b-1b0e-4159-98d3-8011326d8a90 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:32.007194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f662b-1b0e-4159-98d3-8011326d8a90 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:32.007194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f662b-1b0e-4159-98d3-8011326d8a90 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:32.007194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 204/400] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:32.011788 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e76ca-f7fc-4f13-9705-3a73485f157b None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:32.012153 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e76ca-f7fc-4f13-9705-3a73485f157b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:32.012432 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e76ca-f7fc-4f13-9705-3a73485f157b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:32.016312 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7fad5-8087-4576-a8e0-24d9955c993a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:32.016312 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7fad5-8087-4576-a8e0-24d9955c993a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:32.016312 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7fad5-8087-4576-a8e0-24d9955c993a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:32.016312 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7fad5-8087-4576-a8e0-24d9955c993a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:32.032901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-fa5e76ca-f7fc-4f13-9705-3a73485f157b None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:32.034321 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-fa5e76ca-f7fc-4f13-9705-3a73485f157b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:32.035445 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7fad5-8087-4576-a8e0-24d9955c993a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:32.044629 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-acb7fad5-8087-4576-a8e0-24d9955c993a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:53:32.044629 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:32.044629 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:32.044629 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:32.044629 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:32.044629 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:32.044629 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:32.044629 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:32.044629 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:32.044629 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:32.044629 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:32.044629 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:32.044629 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:32.044629 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:32.044629 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:32.044629 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:32.046836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:32.046836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:32.046836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:32.046836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:32.046836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:32.046836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:32.046836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:32.046836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:32.046836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:32.046836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:32.046836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:32.046836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:32.046836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:32.046836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:32.046836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:32.046836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:32.048913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:32.048913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:32.048913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:32.048913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:32.048913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:32.048913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:32.048913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:32.048913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:32.048913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:32.048913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:32.048913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:53:32.048913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:53:32.048913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 197/401] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:53:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:32.048913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7fad5-8087-4576-a8e0-24d9955c993a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:32.132424 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 205/402] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:53:32.133715 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e76ca-f7fc-4f13-9705-3a73485f157b None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:32.200754 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7fad5-8087-4576-a8e0-24d9955c993a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:32.245314 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5620a9f-ed3d-450e-8282-37fcbc5096db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:32.245961 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5620a9f-ed3d-450e-8282-37fcbc5096db None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:32.246237 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5620a9f-ed3d-450e-8282-37fcbc5096db None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:32.246508 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5620a9f-ed3d-450e-8282-37fcbc5096db None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:32.247297 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5620a9f-ed3d-450e-8282-37fcbc5096db None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:32.247602 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5620a9f-ed3d-450e-8282-37fcbc5096db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:32.248737 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:32.249088 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5620a9f-ed3d-450e-8282-37fcbc5096db None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:32.258805 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 198/403] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:53:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:32.264114 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5620a9f-ed3d-450e-8282-37fcbc5096db None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:32.286873 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:32.319751 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e76ca-f7fc-4f13-9705-3a73485f157b None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:32.398875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37447d7e-0fa1-4f52-b77e-fbbbaeda3a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:32.400725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37447d7e-0fa1-4f52-b77e-fbbbaeda3a51 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:32.401429 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37447d7e-0fa1-4f52-b77e-fbbbaeda3a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:32.402306 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37447d7e-0fa1-4f52-b77e-fbbbaeda3a51 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:32.404651 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37447d7e-0fa1-4f52-b77e-fbbbaeda3a51 None admin] RBAC: Adding query filter params (name=member) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:32.405377 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37447d7e-0fa1-4f52-b77e-fbbbaeda3a51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:32.406052 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5620a9f-ed3d-450e-8282-37fcbc5096db None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:32.408375 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37447d7e-0fa1-4f52-b77e-fbbbaeda3a51 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:32.424419 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 206/404] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 13:53:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:32.425991 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37447d7e-0fa1-4f52-b77e-fbbbaeda3a51 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:32.454584 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934efda5-b23d-43b6-bac8-b3ac30d0c56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:32.455422 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934efda5-b23d-43b6-bac8-b3ac30d0c56a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:32.455703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934efda5-b23d-43b6-bac8-b3ac30d0c56a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:32.455981 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934efda5-b23d-43b6-bac8-b3ac30d0c56a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:32.471603 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934efda5-b23d-43b6-bac8-b3ac30d0c56a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:32.478491 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-934efda5-b23d-43b6-bac8-b3ac30d0c56a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:53:32.478491 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:32.478491 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:32.478491 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:32.478491 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:32.478491 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:32.478491 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:32.478491 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:32.478491 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:32.478491 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:32.478491 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:32.478491 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:32.478491 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:32.478491 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:32.478491 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:32.478491 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:32.479448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:32.479448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:32.479448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:32.479448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:32.479448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:32.479448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:32.479448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:32.479448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:32.479448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:32.479448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:32.479448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:32.479448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:32.479448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:32.479448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:32.479448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:32.479448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:32.480342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:32.480342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:32.480342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:32.480342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:32.480342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:32.480342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:32.480342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:32.480342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:32.480342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:32.480342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:32.480342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:53:32.480342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:53:32.487999 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 199/405] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 13:53:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:32.487999 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934efda5-b23d-43b6-bac8-b3ac30d0c56a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:32.568361 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37447d7e-0fa1-4f52-b77e-fbbbaeda3a51 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:32.583014 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:32.607768 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:32.615874 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937fc847-4f63-4a87-bd52-5db30df4fdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:32.616720 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fc847-4f63-4a87-bd52-5db30df4fdf2 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:32.617138 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fc847-4f63-4a87-bd52-5db30df4fdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:32.617420 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fc847-4f63-4a87-bd52-5db30df4fdf2 None admin] PATH_INFO: `/v3/groups/a8b1953358064ba4b85be4020d28d938` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:32.618757 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937fc847-4f63-4a87-bd52-5db30df4fdf2 None admin] RBAC: Authorizing `identity:get_group(group_id=a8b1953358064ba4b85be4020d28d938)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:32.621021 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937fc847-4f63-4a87-bd52-5db30df4fdf2 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:32.622758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934efda5-b23d-43b6-bac8-b3ac30d0c56a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:32.622875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 207/406] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Aug 1 13:53:32 2022] GET /identity/v3/groups/a8b1953358064ba4b85be4020d28d938 => generated 229 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:32.623555 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937fc847-4f63-4a87-bd52-5db30df4fdf2 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:32.667437 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7707bf1e-a508-4de7-b36d-a200ae90ceeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:32.668103 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707bf1e-a508-4de7-b36d-a200ae90ceeb None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:32.668416 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707bf1e-a508-4de7-b36d-a200ae90ceeb None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:32.668698 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707bf1e-a508-4de7-b36d-a200ae90ceeb None admin] PATH_INFO: `/v3/groups/6074835adfc045b496c12ccdf45c7672` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:32.669963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7707bf1e-a508-4de7-b36d-a200ae90ceeb None admin] RBAC: Authorizing `identity:get_group(group_id=6074835adfc045b496c12ccdf45c7672)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:32.672791 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7707bf1e-a508-4de7-b36d-a200ae90ceeb None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:32.677691 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 200/407] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Aug 1 13:53:32 2022] GET /identity/v3/groups/6074835adfc045b496c12ccdf45c7672 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:32.678688 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7707bf1e-a508-4de7-b36d-a200ae90ceeb None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:32.774131 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937fc847-4f63-4a87-bd52-5db30df4fdf2 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:32.818100 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7707bf1e-a508-4de7-b36d-a200ae90ceeb None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:32.824871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7debc05f-c39f-425e-a45c-138c9c2d48e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:32.825866 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7debc05f-c39f-425e-a45c-138c9c2d48e1 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:32.826365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7debc05f-c39f-425e-a45c-138c9c2d48e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:32.826954 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7debc05f-c39f-425e-a45c-138c9c2d48e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:32.828172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7debc05f-c39f-425e-a45c-138c9c2d48e1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:32.828722 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7debc05f-c39f-425e-a45c-138c9c2d48e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:32.830530 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7debc05f-c39f-425e-a45c-138c9c2d48e1 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:32.844930 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 208/408] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:53:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:32.853985 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7debc05f-c39f-425e-a45c-138c9c2d48e1 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:32.860123 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656addb2-b144-4c78-a9e1-3c19112a7538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:32.860933 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656addb2-b144-4c78-a9e1-3c19112a7538 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:32.862278 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656addb2-b144-4c78-a9e1-3c19112a7538 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:32.862671 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656addb2-b144-4c78-a9e1-3c19112a7538 None admin] PATH_INFO: `/v3/projects/2a12613c4ac44ad788edb767368b347b` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:32.864069 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656addb2-b144-4c78-a9e1-3c19112a7538 None admin] RBAC: Authorizing `identity:get_project(project_id=2a12613c4ac44ad788edb767368b347b)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:32.865771 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656addb2-b144-4c78-a9e1-3c19112a7538 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:32.867870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 201/409] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:53:32 2022] GET /identity/v3/projects/2a12613c4ac44ad788edb767368b347b => generated 344 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:32.868536 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656addb2-b144-4c78-a9e1-3c19112a7538 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:32.881007 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:32.945959 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:33.000779 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656addb2-b144-4c78-a9e1-3c19112a7538 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:33.025873 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7debc05f-c39f-425e-a45c-138c9c2d48e1 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:33.067437 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d2eca-f024-44b7-aae4-abc62de350cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:33.068422 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d2eca-f024-44b7-aae4-abc62de350cf None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:33.068966 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d2eca-f024-44b7-aae4-abc62de350cf None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:33.069400 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d2eca-f024-44b7-aae4-abc62de350cf None admin] PATH_INFO: `/v3/projects/7489ab03e4a8475cab57be63c14ec0c0` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:33.070926 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d2eca-f024-44b7-aae4-abc62de350cf None admin] RBAC: Authorizing `identity:get_project(project_id=7489ab03e4a8475cab57be63c14ec0c0)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:33.072610 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d2eca-f024-44b7-aae4-abc62de350cf None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:33.074905 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 209/410] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:53:32 2022] GET /identity/v3/projects/7489ab03e4a8475cab57be63c14ec0c0 => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:33.075894 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d2eca-f024-44b7-aae4-abc62de350cf None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:33.081017 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e85500-58f4-410d-b7a7-f97193295ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:33.082045 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e85500-58f4-410d-b7a7-f97193295ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:33.082516 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e85500-58f4-410d-b7a7-f97193295ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:33.082788 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e85500-58f4-410d-b7a7-f97193295ac9 None admin] PATH_INFO: `/v3/groups/6074835adfc045b496c12ccdf45c7672` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:33.084044 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e85500-58f4-410d-b7a7-f97193295ac9 None admin] RBAC: Authorizing `identity:get_group(group_id=6074835adfc045b496c12ccdf45c7672)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:33.085776 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e85500-58f4-410d-b7a7-f97193295ac9 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:33.087843 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 202/411] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Aug 1 13:53:32 2022] GET /identity/v3/groups/6074835adfc045b496c12ccdf45c7672 => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:33.095359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e85500-58f4-410d-b7a7-f97193295ac9 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:33.197569 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:33.230513 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e85500-58f4-410d-b7a7-f97193295ac9 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:33.240742 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d2eca-f024-44b7-aae4-abc62de350cf None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:33.274352 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:33.278868 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2a46f-86bb-4c39-a53b-769b56039aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:33.279853 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2a46f-86bb-4c39-a53b-769b56039aaf None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:33.280244 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2a46f-86bb-4c39-a53b-769b56039aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:33.280674 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2a46f-86bb-4c39-a53b-769b56039aaf None admin] PATH_INFO: `/v3/projects/7489ab03e4a8475cab57be63c14ec0c0/groups/6074835adfc045b496c12ccdf45c7672/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:33.283966 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b2a46f-86bb-4c39-a53b-769b56039aaf None admin] RBAC: Authorizing `identity:create_grant(project_id=7489ab03e4a8475cab57be63c14ec0c0, group_id=6074835adfc045b496c12ccdf45c7672, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:33.286272 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b2a46f-86bb-4c39-a53b-769b56039aaf None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:33.286930 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7da3d-54a2-447d-be9a-a2136507bf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:33.287796 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7da3d-54a2-447d-be9a-a2136507bf24 None admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:33.288232 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7da3d-54a2-447d-be9a-a2136507bf24 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:33.288714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7da3d-54a2-447d-be9a-a2136507bf24 None admin] PATH_INFO: `/v3/projects/2a12613c4ac44ad788edb767368b347b/groups/a8b1953358064ba4b85be4020d28d938/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:33.291232 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7da3d-54a2-447d-be9a-a2136507bf24 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a12613c4ac44ad788edb767368b347b, group_id=a8b1953358064ba4b85be4020d28d938, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:33.293288 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7da3d-54a2-447d-be9a-a2136507bf24 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:33.307142 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 203/412] 10.222.0.45 () {58 vars in 1442 bytes} [Mon Aug 1 13:53:33 2022] PUT /identity/v3/projects/7489ab03e4a8475cab57be63c14ec0c0/groups/6074835adfc045b496c12ccdf45c7672/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:53:33.308038 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2a46f-86bb-4c39-a53b-769b56039aaf None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:33.315987 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 210/413] 10.222.0.45 () {58 vars in 1442 bytes} [Mon Aug 1 13:53:33 2022] PUT /identity/v3/projects/2a12613c4ac44ad788edb767368b347b/groups/a8b1953358064ba4b85be4020d28d938/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:53:33.382866 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e9baa-18e4-442d-8b64-15b15dc624a6 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:33.383508 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e9baa-18e4-442d-8b64-15b15dc624a6 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:33.383980 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e9baa-18e4-442d-8b64-15b15dc624a6 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:33.386419 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 211/414] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:33.394530 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc32981-af29-49e7-8ec4-16b67d51fdeb None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:33.395219 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc32981-af29-49e7-8ec4-16b67d51fdeb None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:33.395688 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc32981-af29-49e7-8ec4-16b67d51fdeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:33.444552 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2a46f-86bb-4c39-a53b-769b56039aaf None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:33.448105 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-1dc32981-af29-49e7-8ec4-16b67d51fdeb None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:33.451189 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1dc32981-af29-49e7-8ec4-16b67d51fdeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:33.489473 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff6026-9951-4836-9d6e-58fc3ccca538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:33.490402 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff6026-9951-4836-9d6e-58fc3ccca538 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:33.490510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff6026-9951-4836-9d6e-58fc3ccca538 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:33.490719 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff6026-9951-4836-9d6e-58fc3ccca538 None admin] PATH_INFO: `/v3/projects/7489ab03e4a8475cab57be63c14ec0c0` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:33.492150 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ff6026-9951-4836-9d6e-58fc3ccca538 None admin] RBAC: Authorizing `identity:get_project(project_id=7489ab03e4a8475cab57be63c14ec0c0)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:33.493817 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ff6026-9951-4836-9d6e-58fc3ccca538 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:33.495892 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 204/415] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:53:33 2022] GET /identity/v3/projects/7489ab03e4a8475cab57be63c14ec0c0 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:33.503400 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff6026-9951-4836-9d6e-58fc3ccca538 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:33.507183 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:33.609563 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 212/416] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:53:33.611540 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d4183-2bb6-42bf-b24a-1d7253c6a8d0 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:33.611841 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d4183-2bb6-42bf-b24a-1d7253c6a8d0 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:33.612119 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d4183-2bb6-42bf-b24a-1d7253c6a8d0 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:33.614159 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 213/417] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:33.619790 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf76a0-b22c-4d59-aee1-d2cc4eca021a None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:33.620084 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf76a0-b22c-4d59-aee1-d2cc4eca021a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:33.620488 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf76a0-b22c-4d59-aee1-d2cc4eca021a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:33.641245 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-a0bf76a0-b22c-4d59-aee1-d2cc4eca021a None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:33.642500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a0bf76a0-b22c-4d59-aee1-d2cc4eca021a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:33.659420 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff6026-9951-4836-9d6e-58fc3ccca538 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:33.707822 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064fe11c-5e7f-4e9b-9ecf-b36e8a5e002c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:33.708720 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fe11c-5e7f-4e9b-9ecf-b36e8a5e002c None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:33.709005 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fe11c-5e7f-4e9b-9ecf-b36e8a5e002c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:33.709278 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fe11c-5e7f-4e9b-9ecf-b36e8a5e002c None admin] PATH_INFO: `/v3/projects/7489ab03e4a8475cab57be63c14ec0c0/groups/6074835adfc045b496c12ccdf45c7672/roles/60efb70fa7354ebfb04aa5e9bdad2a78` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:33.711674 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064fe11c-5e7f-4e9b-9ecf-b36e8a5e002c None admin] RBAC: Authorizing `identity:create_grant(project_id=7489ab03e4a8475cab57be63c14ec0c0, group_id=6074835adfc045b496c12ccdf45c7672, role_id=60efb70fa7354ebfb04aa5e9bdad2a78)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:33.713450 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064fe11c-5e7f-4e9b-9ecf-b36e8a5e002c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:33.734265 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 205/418] 10.222.0.45 () {58 vars in 1442 bytes} [Mon Aug 1 13:53:33 2022] PUT /identity/v3/projects/7489ab03e4a8475cab57be63c14ec0c0/groups/6074835adfc045b496c12ccdf45c7672/roles/60efb70fa7354ebfb04aa5e9bdad2a78 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 13:53:33.736736 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064fe11c-5e7f-4e9b-9ecf-b36e8a5e002c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:33.753907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 214/419] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:33.859443 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:33.861953 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf76a0-b22c-4d59-aee1-d2cc4eca021a None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:33.925330 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064fe11c-5e7f-4e9b-9ecf-b36e8a5e002c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:33.982537 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734f1e4-e554-4d27-910b-d566c98b91a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:33.983556 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734f1e4-e554-4d27-910b-d566c98b91a2 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:33.984891 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734f1e4-e554-4d27-910b-d566c98b91a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:33.985361 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734f1e4-e554-4d27-910b-d566c98b91a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:33.986391 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:34.008480 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5734f1e4-e554-4d27-910b-d566c98b91a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:34.019800 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-5734f1e4-e554-4d27-910b-d566c98b91a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:53:34.019800 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:34.019800 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:34.019800 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:34.019800 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:34.019800 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:34.019800 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:34.019800 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:34.019800 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:34.019800 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:34.019800 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:34.019800 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:34.019800 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:34.019800 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:34.019800 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:34.019800 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:34.021426 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:34.021426 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:34.021426 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:34.021426 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:34.021426 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:34.021426 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:34.021426 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:34.021426 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:34.021426 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:34.021426 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:34.021426 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:34.021426 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:34.021426 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:34.021426 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:34.021426 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:34.021426 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:34.022795 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:34.022795 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:34.022795 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:34.022795 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:34.022795 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:34.022795 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:34.022795 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:34.022795 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:34.022795 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:34.022795 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:34.022795 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:53:34.022795 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:53:34.022795 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 206/420] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:53:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:34.029444 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734f1e4-e554-4d27-910b-d566c98b91a2 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:34.081307 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf76a0-b22c-4d59-aee1-d2cc4eca021a None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:34.140857 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68280507-4f78-4c97-a1bf-e160c69a6056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:34.141984 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68280507-4f78-4c97-a1bf-e160c69a6056 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:34.142500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68280507-4f78-4c97-a1bf-e160c69a6056 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:34.142949 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68280507-4f78-4c97-a1bf-e160c69a6056 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:34.159358 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68280507-4f78-4c97-a1bf-e160c69a6056 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:34.166146 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-68280507-4f78-4c97-a1bf-e160c69a6056 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:53:34.166146 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:34.166146 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:34.166146 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:34.166146 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:34.166146 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:34.166146 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:34.166146 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:34.166146 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:34.166146 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:34.166146 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:34.166146 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:34.166146 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:34.166146 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:34.166146 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:34.166146 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:34.167729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:34.167729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:34.167729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:34.167729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:34.167729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:34.167729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:34.167729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:34.167729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:34.167729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:34.167729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:34.167729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:34.167729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:34.167729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:34.167729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:34.167729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:34.167729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:34.169061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:34.169061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:34.169061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:34.169061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:34.169061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:34.169061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:34.169061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:34.169061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:34.169061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:34.169061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:34.169061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:53:34.169061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:34.171935 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 215/421] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 13:53:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:53:34.178733 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68280507-4f78-4c97-a1bf-e160c69a6056 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:34.230261 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734f1e4-e554-4d27-910b-d566c98b91a2 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:34.273497 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:34.298485 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d40a0d-d124-461a-83a6-7385409154fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:34.299260 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d40a0d-d124-461a-83a6-7385409154fc None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:34.299598 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d40a0d-d124-461a-83a6-7385409154fc None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:34.300038 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d40a0d-d124-461a-83a6-7385409154fc None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:34.300939 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d40a0d-d124-461a-83a6-7385409154fc None admin] RBAC: Adding query filter params (name=member) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:34.301249 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d40a0d-d124-461a-83a6-7385409154fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:34.305270 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d40a0d-d124-461a-83a6-7385409154fc None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:34.318076 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 207/422] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 13:53:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:53:34.329012 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d40a0d-d124-461a-83a6-7385409154fc None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:34.399085 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68280507-4f78-4c97-a1bf-e160c69a6056 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:34.424105 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:34.476954 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46984355-80c7-406a-a04a-981cc3f53534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:34.478636 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46984355-80c7-406a-a04a-981cc3f53534 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:34.479418 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46984355-80c7-406a-a04a-981cc3f53534 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:34.479931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46984355-80c7-406a-a04a-981cc3f53534 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:34.481198 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46984355-80c7-406a-a04a-981cc3f53534 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:34.481838 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46984355-80c7-406a-a04a-981cc3f53534 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:34.484227 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46984355-80c7-406a-a04a-981cc3f53534 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:34.500548 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 216/423] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:53:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:34.503229 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46984355-80c7-406a-a04a-981cc3f53534 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:34.536057 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d40a0d-d124-461a-83a6-7385409154fc None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:34.655303 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46984355-80c7-406a-a04a-981cc3f53534 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:34.677925 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4419d0ac-2099-4e01-a900-77cc74622e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:34.678580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419d0ac-2099-4e01-a900-77cc74622e79 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:34.678862 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419d0ac-2099-4e01-a900-77cc74622e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:34.679133 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419d0ac-2099-4e01-a900-77cc74622e79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:34.708702 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602969b2-a424-49d6-9065-68329e30a711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:34.710107 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602969b2-a424-49d6-9065-68329e30a711 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:34.710900 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602969b2-a424-49d6-9065-68329e30a711 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:34.711655 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602969b2-a424-49d6-9065-68329e30a711 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:34.797082 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:34.862639 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602969b2-a424-49d6-9065-68329e30a711 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:34.864816 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602969b2-a424-49d6-9065-68329e30a711 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:34.879083 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:34.961385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4419d0ac-2099-4e01-a900-77cc74622e79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:34.962897 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4419d0ac-2099-4e01-a900-77cc74622e79 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:35.002937 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 217/424] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:53:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 13:53:35.009268 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602969b2-a424-49d6-9065-68329e30a711 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:35.107574 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:35.137481 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602969b2-a424-49d6-9065-68329e30a711 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:35.185998 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d767e18a-c763-4fc3-89c0-a71067c1faf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:35.187699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d767e18a-c763-4fc3-89c0-a71067c1faf8 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:35.188591 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d767e18a-c763-4fc3-89c0-a71067c1faf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:35.189589 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d767e18a-c763-4fc3-89c0-a71067c1faf8 None admin] PATH_INFO: `/v3/projects/c18388255f054822b773ec88708292da` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:35.193070 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d767e18a-c763-4fc3-89c0-a71067c1faf8 None admin] RBAC: Authorizing `identity:get_project(project_id=c18388255f054822b773ec88708292da)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:35.197349 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d767e18a-c763-4fc3-89c0-a71067c1faf8 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:35.202317 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 218/425] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:53:35 2022] GET /identity/v3/projects/c18388255f054822b773ec88708292da => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:35.212515 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d767e18a-c763-4fc3-89c0-a71067c1faf8 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:35.281118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 208/426] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:53:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 953 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:53:35.289815 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4419d0ac-2099-4e01-a900-77cc74622e79 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:35.458573 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d767e18a-c763-4fc3-89c0-a71067c1faf8 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:35.552606 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:35.561460 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba70e8-1e1c-4d29-9864-0901af9ce4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:35.564079 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba70e8-1e1c-4d29-9864-0901af9ce4c0 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:35.564893 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba70e8-1e1c-4d29-9864-0901af9ce4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:35.565788 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba70e8-1e1c-4d29-9864-0901af9ce4c0 None admin] PATH_INFO: `/v3/groups/6074835adfc045b496c12ccdf45c7672` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:35.568937 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ba70e8-1e1c-4d29-9864-0901af9ce4c0 None admin] RBAC: Authorizing `identity:get_group(group_id=6074835adfc045b496c12ccdf45c7672)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:35.572872 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ba70e8-1e1c-4d29-9864-0901af9ce4c0 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:35.574575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4419d0ac-2099-4e01-a900-77cc74622e79 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:35.578579 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 219/427] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Aug 1 13:53:35 2022] GET /identity/v3/groups/6074835adfc045b496c12ccdf45c7672 => generated 226 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:35.599807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba70e8-1e1c-4d29-9864-0901af9ce4c0 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:35.625090 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:35.718911 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb251af-7d46-4de3-b06c-8dc9a35d9e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:35.719930 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb251af-7d46-4de3-b06c-8dc9a35d9e9c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:35.720429 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb251af-7d46-4de3-b06c-8dc9a35d9e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:35.720893 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb251af-7d46-4de3-b06c-8dc9a35d9e9c None admin] PATH_INFO: `/v3/projects/c18388255f054822b773ec88708292da` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:35.730048 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb251af-7d46-4de3-b06c-8dc9a35d9e9c None admin] RBAC: Authorizing `identity:get_project(project_id=c18388255f054822b773ec88708292da)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:35.732527 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb251af-7d46-4de3-b06c-8dc9a35d9e9c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:35.736102 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 209/428] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:53:35 2022] GET /identity/v3/projects/c18388255f054822b773ec88708292da => generated 298 bytes in 447 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:53:35.744623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb251af-7d46-4de3-b06c-8dc9a35d9e9c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:35.829690 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba70e8-1e1c-4d29-9864-0901af9ce4c0 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:35.902950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c829b3b-88ea-4420-9e2a-ee81c5bb0080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:35.904739 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c829b3b-88ea-4420-9e2a-ee81c5bb0080 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:35.905731 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c829b3b-88ea-4420-9e2a-ee81c5bb0080 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:35.906580 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c829b3b-88ea-4420-9e2a-ee81c5bb0080 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:35.908564 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c829b3b-88ea-4420-9e2a-ee81c5bb0080 None admin] RBAC: Adding query filter params (group.id=6074835adfc045b496c12ccdf45c7672, role.id=60efb70fa7354ebfb04aa5e9bdad2a78, scope.project.id=c18388255f054822b773ec88708292da) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:35.909475 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c829b3b-88ea-4420-9e2a-ee81c5bb0080 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:35.912941 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c829b3b-88ea-4420-9e2a-ee81c5bb0080 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:35.925337 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 220/429] 10.222.0.45 () {56 vars in 1376 bytes} [Mon Aug 1 13:53:35 2022] GET /identity/v3/role_assignments?group.id=6074835adfc045b496c12ccdf45c7672&scope.project.id=c18388255f054822b773ec88708292da&role.id=60efb70fa7354ebfb04aa5e9bdad2a78 => generated 611 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:36.099880 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb251af-7d46-4de3-b06c-8dc9a35d9e9c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:36.177693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2babd2-0f38-43d6-94b7-43b26cbd7573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:36.178800 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2babd2-0f38-43d6-94b7-43b26cbd7573 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:36.179278 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2babd2-0f38-43d6-94b7-43b26cbd7573 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:36.179740 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2babd2-0f38-43d6-94b7-43b26cbd7573 None admin] PATH_INFO: `/v3/groups/6074835adfc045b496c12ccdf45c7672` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:36.185740 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2babd2-0f38-43d6-94b7-43b26cbd7573 None admin] RBAC: Authorizing `identity:get_group(group_id=6074835adfc045b496c12ccdf45c7672)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:36.188566 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2babd2-0f38-43d6-94b7-43b26cbd7573 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:36.192071 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 210/430] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Aug 1 13:53:35 2022] GET /identity/v3/groups/6074835adfc045b496c12ccdf45c7672 => generated 226 bytes in 448 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:36.201122 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c829b3b-88ea-4420-9e2a-ee81c5bb0080 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:36.343971 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:36.384111 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c829b3b-88ea-4420-9e2a-ee81c5bb0080 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:36.438978 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e4c9bb-4c09-425f-9aff-35b58ffd777a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:36.440044 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4c9bb-4c09-425f-9aff-35b58ffd777a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:36.440202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4c9bb-4c09-425f-9aff-35b58ffd777a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:36.440620 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4c9bb-4c09-425f-9aff-35b58ffd777a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:36.441915 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e4c9bb-4c09-425f-9aff-35b58ffd777a None admin] RBAC: Adding query filter params (group.id=6074835adfc045b496c12ccdf45c7672, role.id=75aff8cb4cf748a2adbd6c3f141fe8ac, scope.project.id=c18388255f054822b773ec88708292da) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:36.442315 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61dae4d-ef23-4c61-96de-33f47fb92a09 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:36.442500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e4c9bb-4c09-425f-9aff-35b58ffd777a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:36.442763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61dae4d-ef23-4c61-96de-33f47fb92a09 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:36.443209 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61dae4d-ef23-4c61-96de-33f47fb92a09 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:36.444214 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e4c9bb-4c09-425f-9aff-35b58ffd777a None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:36.446827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 211/431] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:36.453915 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 221/432] 10.222.0.45 () {56 vars in 1376 bytes} [Mon Aug 1 13:53:36 2022] GET /identity/v3/role_assignments?group.id=6074835adfc045b496c12ccdf45c7672&scope.project.id=c18388255f054822b773ec88708292da&role.id=75aff8cb4cf748a2adbd6c3f141fe8ac => generated 611 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:36.455858 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c66fed-1064-4cc5-b9d4-94384d810260 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:36.459749 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c66fed-1064-4cc5-b9d4-94384d810260 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:36.459749 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c66fed-1064-4cc5-b9d4-94384d810260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:36.487976 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-58c66fed-1064-4cc5-b9d4-94384d810260 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:36.490011 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-58c66fed-1064-4cc5-b9d4-94384d810260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:36.515894 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:36.592224 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 212/433] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:53:36.624188 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e4c9bb-4c09-425f-9aff-35b58ffd777a None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:36.631805 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92a5d2-4062-4a2a-8946-ca0efdbc4408 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:36.632170 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92a5d2-4062-4a2a-8946-ca0efdbc4408 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:36.632399 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92a5d2-4062-4a2a-8946-ca0efdbc4408 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:36.635100 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 213/434] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:36.646330 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f75877-5d6b-4fec-af83-656dd52e4912 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:36.646678 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f75877-5d6b-4fec-af83-656dd52e4912 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:36.646967 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f75877-5d6b-4fec-af83-656dd52e4912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:36.667287 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a5f75877-5d6b-4fec-af83-656dd52e4912 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:36.669138 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a5f75877-5d6b-4fec-af83-656dd52e4912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:36.811602 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 214/435] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:53:36.814944 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234e018-9957-43f5-ad81-9be0f0ad2729 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:36.815728 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234e018-9957-43f5-ad81-9be0f0ad2729 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:36.816421 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234e018-9957-43f5-ad81-9be0f0ad2729 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:36.818660 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 215/436] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:36.827013 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b442f-b8d7-4881-ac74-f999227b828e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:36.827691 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b442f-b8d7-4881-ac74-f999227b828e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:36.828232 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b442f-b8d7-4881-ac74-f999227b828e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:36.842489 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e4c9bb-4c09-425f-9aff-35b58ffd777a None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:36.866725 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-019b442f-b8d7-4881-ac74-f999227b828e None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:36.866981 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-019b442f-b8d7-4881-ac74-f999227b828e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:36.907374 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff84c65-3647-4592-8b3a-8c170bbfd29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:36.908379 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff84c65-3647-4592-8b3a-8c170bbfd29d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:36.908869 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff84c65-3647-4592-8b3a-8c170bbfd29d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:36.909338 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff84c65-3647-4592-8b3a-8c170bbfd29d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:36.912782 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:36.937093 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff84c65-3647-4592-8b3a-8c170bbfd29d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:36.950219 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-aff84c65-3647-4592-8b3a-8c170bbfd29d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:53:36.950219 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:36.950219 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:36.950219 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:36.950219 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:36.950219 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:36.950219 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:36.950219 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:36.950219 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:36.950219 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:36.950219 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:36.950219 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:36.950219 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:36.950219 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:36.950219 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:36.950219 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:36.951578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:36.951578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:36.951578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:36.951578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:36.951578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:36.951578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:36.951578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:36.951578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:36.951578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:36.951578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:36.951578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:36.951578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:36.951578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:36.951578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:36.951578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:36.951578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:36.953135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:36.953135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:36.953135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:36.953135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:36.953135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:36.953135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:36.953135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:36.953135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:36.953135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:36.953135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:36.953135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:53:36.953135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:36.953135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 222/437] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:53:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:36.954400 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff84c65-3647-4592-8b3a-8c170bbfd29d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:37.005650 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 216/438] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:53:37.005650 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b442f-b8d7-4881-ac74-f999227b828e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:37.024520 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:37.182082 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff84c65-3647-4592-8b3a-8c170bbfd29d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:37.218873 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b442f-b8d7-4881-ac74-f999227b828e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:37.261097 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5a827-1c5a-4cf6-a6a4-12b5adb9507a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:37.262200 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5a827-1c5a-4cf6-a6a4-12b5adb9507a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:37.262675 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5a827-1c5a-4cf6-a6a4-12b5adb9507a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:37.263173 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5a827-1c5a-4cf6-a6a4-12b5adb9507a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:37.284032 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f5a827-1c5a-4cf6-a6a4-12b5adb9507a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:37.292665 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-a3f5a827-1c5a-4cf6-a6a4-12b5adb9507a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:53:37.292665 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:37.292665 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:37.292665 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:37.292665 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:37.292665 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:37.292665 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:37.292665 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:37.292665 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:37.292665 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:37.292665 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:37.292665 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:37.292665 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:37.292665 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:37.292665 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:37.292665 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:37.294470 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:37.294470 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:37.294470 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:37.294470 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:37.294470 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:37.294470 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:37.294470 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:37.294470 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:37.294470 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:37.294470 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:37.294470 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:37.294470 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:37.294470 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:37.294470 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:37.294470 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:37.294470 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:37.296017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:37.296017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:37.296017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:37.296017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:37.296017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:37.296017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:37.296017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:37.296017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:37.296017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:37.296017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:37.296017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:53:37.296017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:37.296017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 223/439] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:53:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:37.296017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5a827-1c5a-4cf6-a6a4-12b5adb9507a None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:37.309108 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4997121-25a7-452c-b445-a05a7ce959b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:37.309108 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4997121-25a7-452c-b445-a05a7ce959b8 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:37.310778 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4997121-25a7-452c-b445-a05a7ce959b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:37.311157 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4997121-25a7-452c-b445-a05a7ce959b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:37.315471 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4997121-25a7-452c-b445-a05a7ce959b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:37.315471 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4997121-25a7-452c-b445-a05a7ce959b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:37.317347 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4997121-25a7-452c-b445-a05a7ce959b8 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:37.334857 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 217/440] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 13:53:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:37.336827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4997121-25a7-452c-b445-a05a7ce959b8 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:37.409732 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:37.462835 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:37.495113 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5a827-1c5a-4cf6-a6a4-12b5adb9507a None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:37.521207 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4997121-25a7-452c-b445-a05a7ce959b8 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:37.545988 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46631f6-1657-4612-9499-168f17ea9688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:37.548312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46631f6-1657-4612-9499-168f17ea9688 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:37.548312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46631f6-1657-4612-9499-168f17ea9688 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:37.548312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46631f6-1657-4612-9499-168f17ea9688 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:37.561636 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46631f6-1657-4612-9499-168f17ea9688 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:37.567141 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-a46631f6-1657-4612-9499-168f17ea9688 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:53:37.567141 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:37.567141 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:37.567141 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:37.567141 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:37.567141 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:37.567141 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:37.567141 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:37.567141 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:37.567141 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:37.567141 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:37.567141 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:37.567141 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:37.567141 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:37.567141 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:37.567141 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:37.568817 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:37.568817 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:37.568817 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:37.568817 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:37.568817 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:37.568817 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:37.568817 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:37.568817 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:37.568817 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:37.568817 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:37.568817 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:37.568817 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:37.568817 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:37.568817 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:37.568817 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:37.568817 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:37.570283 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:37.570283 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:37.570283 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:37.570283 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:37.570283 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:37.570283 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:37.570283 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:37.570283 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:37.570283 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:37.570283 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:37.570283 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:53:37.570283 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:37.570283 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 224/441] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 13:53:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:37.570283 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46631f6-1657-4612-9499-168f17ea9688 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:37.578342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1c634-ee59-4587-865f-675ec1208888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:37.580707 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1c634-ee59-4587-865f-675ec1208888 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:37.580707 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1c634-ee59-4587-865f-675ec1208888 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:37.580707 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1c634-ee59-4587-865f-675ec1208888 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:37.581304 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a1c634-ee59-4587-865f-675ec1208888 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:37.581304 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a1c634-ee59-4587-865f-675ec1208888 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:37.583596 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a1c634-ee59-4587-865f-675ec1208888 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:37.596145 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 218/442] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:53:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:37.601613 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1c634-ee59-4587-865f-675ec1208888 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:37.731669 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46631f6-1657-4612-9499-168f17ea9688 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:37.759185 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:37.783471 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1c634-ee59-4587-865f-675ec1208888 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:37.794996 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d89d8-869c-479e-a660-7d7f835740e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:37.796023 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d89d8-869c-479e-a660-7d7f835740e9 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:37.796648 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d89d8-869c-479e-a660-7d7f835740e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:37.797170 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d89d8-869c-479e-a660-7d7f835740e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:37.833368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca844683-3319-46c3-8197-079949c47603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:37.834215 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca844683-3319-46c3-8197-079949c47603 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:37.834509 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca844683-3319-46c3-8197-079949c47603 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:37.834893 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca844683-3319-46c3-8197-079949c47603 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:37.835914 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca844683-3319-46c3-8197-079949c47603 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:37.836254 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca844683-3319-46c3-8197-079949c47603 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:37.837724 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca844683-3319-46c3-8197-079949c47603 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:37.840024 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:37.851885 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 219/443] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:53:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:37.852871 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca844683-3319-46c3-8197-079949c47603 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:37.926170 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d89d8-869c-479e-a660-7d7f835740e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:37.928000 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d89d8-869c-479e-a660-7d7f835740e9 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:38.058500 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca844683-3319-46c3-8197-079949c47603 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:38.108657 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 225/444] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:53:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:53:38.109926 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d89d8-869c-479e-a660-7d7f835740e9 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:38.121851 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de482d7-0e86-4de6-8ce4-1d2f6ff497e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:38.122922 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de482d7-0e86-4de6-8ce4-1d2f6ff497e7 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:38.123427 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de482d7-0e86-4de6-8ce4-1d2f6ff497e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:38.123911 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de482d7-0e86-4de6-8ce4-1d2f6ff497e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:38.131921 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:38.224298 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:38.238306 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de482d7-0e86-4de6-8ce4-1d2f6ff497e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:38.241083 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de482d7-0e86-4de6-8ce4-1d2f6ff497e7 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:38.280791 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d89d8-869c-479e-a660-7d7f835740e9 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:38.353088 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff1d11-430e-41fd-bdd4-db120481a49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:38.354070 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff1d11-430e-41fd-bdd4-db120481a49c None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:38.354567 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff1d11-430e-41fd-bdd4-db120481a49c None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:38.355020 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff1d11-430e-41fd-bdd4-db120481a49c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:38.404155 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 220/445] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:53:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:53:38.405113 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de482d7-0e86-4de6-8ce4-1d2f6ff497e7 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:38.474776 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ff1d11-430e-41fd-bdd4-db120481a49c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:38.476605 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ff1d11-430e-41fd-bdd4-db120481a49c None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:38.503132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:38.582200 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:38.593361 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de482d7-0e86-4de6-8ce4-1d2f6ff497e7 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:38.647590 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 226/446] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:53:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:53:38.648969 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff1d11-430e-41fd-bdd4-db120481a49c None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:38.660745 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e42cb-3632-425a-b1e2-1c378d9f5dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:38.661805 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e42cb-3632-425a-b1e2-1c378d9f5dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:38.662321 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e42cb-3632-425a-b1e2-1c378d9f5dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:38.662763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e42cb-3632-425a-b1e2-1c378d9f5dd3 None admin] PATH_INFO: `/v3/projects/7489ab03e4a8475cab57be63c14ec0c0` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:38.665085 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3e42cb-3632-425a-b1e2-1c378d9f5dd3 None admin] RBAC: Authorizing `identity:get_project(project_id=7489ab03e4a8475cab57be63c14ec0c0)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:38.667497 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3e42cb-3632-425a-b1e2-1c378d9f5dd3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:38.670423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 221/447] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:53:38 2022] GET /identity/v3/projects/7489ab03e4a8475cab57be63c14ec0c0 => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:38.672305 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e42cb-3632-425a-b1e2-1c378d9f5dd3 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:38.804956 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff1d11-430e-41fd-bdd4-db120481a49c None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:38.830106 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:38.845810 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e42cb-3632-425a-b1e2-1c378d9f5dd3 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:38.862560 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276c020-62ce-489d-ac87-5615abea5325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:38.863435 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276c020-62ce-489d-ac87-5615abea5325 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:38.863731 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276c020-62ce-489d-ac87-5615abea5325 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:38.864122 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276c020-62ce-489d-ac87-5615abea5325 None admin] PATH_INFO: `/v3/projects/2a12613c4ac44ad788edb767368b347b` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:38.866285 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b276c020-62ce-489d-ac87-5615abea5325 None admin] RBAC: Authorizing `identity:get_project(project_id=2a12613c4ac44ad788edb767368b347b)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:38.868069 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b276c020-62ce-489d-ac87-5615abea5325 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:38.870220 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 227/448] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:53:38 2022] GET /identity/v3/projects/2a12613c4ac44ad788edb767368b347b => generated 344 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:38.871495 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276c020-62ce-489d-ac87-5615abea5325 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:38.908489 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23215548-8f9a-42e8-a856-676cea3adb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:38.909324 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23215548-8f9a-42e8-a856-676cea3adb8c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:38.909714 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23215548-8f9a-42e8-a856-676cea3adb8c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:38.910081 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23215548-8f9a-42e8-a856-676cea3adb8c None admin] PATH_INFO: `/v3/projects/7489ab03e4a8475cab57be63c14ec0c0` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:38.911562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23215548-8f9a-42e8-a856-676cea3adb8c None admin] RBAC: Authorizing `identity:get_project(project_id=7489ab03e4a8475cab57be63c14ec0c0)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:38.913325 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23215548-8f9a-42e8-a856-676cea3adb8c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:38.915217 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 222/449] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:53:38 2022] GET /identity/v3/projects/7489ab03e4a8475cab57be63c14ec0c0 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:38.916627 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23215548-8f9a-42e8-a856-676cea3adb8c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:38.973032 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:39.045510 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276c020-62ce-489d-ac87-5615abea5325 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:39.080202 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23215548-8f9a-42e8-a856-676cea3adb8c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:39.113194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c2bca-7142-4081-ac0d-2577f042e24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:39.114162 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c2bca-7142-4081-ac0d-2577f042e24d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:39.114449 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c2bca-7142-4081-ac0d-2577f042e24d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:39.114816 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c2bca-7142-4081-ac0d-2577f042e24d None admin] PATH_INFO: `/v3/groups/6074835adfc045b496c12ccdf45c7672` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:39.116623 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c2bca-7142-4081-ac0d-2577f042e24d None admin] RBAC: Authorizing `identity:get_group(group_id=6074835adfc045b496c12ccdf45c7672)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:39.119118 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c2bca-7142-4081-ac0d-2577f042e24d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:39.120885 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 228/450] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Aug 1 13:53:38 2022] GET /identity/v3/groups/6074835adfc045b496c12ccdf45c7672 => generated 226 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:39.121678 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c2bca-7142-4081-ac0d-2577f042e24d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:39.137059 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4948d09-5357-4948-8035-1dc5ccf86bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:39.138399 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4948d09-5357-4948-8035-1dc5ccf86bbf None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:39.138783 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4948d09-5357-4948-8035-1dc5ccf86bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:39.139222 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4948d09-5357-4948-8035-1dc5ccf86bbf None admin] PATH_INFO: `/v3/groups/a8b1953358064ba4b85be4020d28d938` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:39.141147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4948d09-5357-4948-8035-1dc5ccf86bbf None admin] RBAC: Authorizing `identity:get_group(group_id=a8b1953358064ba4b85be4020d28d938)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:39.143872 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4948d09-5357-4948-8035-1dc5ccf86bbf None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:39.146224 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 223/451] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Aug 1 13:53:38 2022] GET /identity/v3/groups/a8b1953358064ba4b85be4020d28d938 => generated 229 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:39.147846 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4948d09-5357-4948-8035-1dc5ccf86bbf None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:39.206825 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:39.303527 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c2bca-7142-4081-ac0d-2577f042e24d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:39.311747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:39.315804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4948d09-5357-4948-8035-1dc5ccf86bbf None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:39.362496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8372b-9c32-4ffd-bb45-7c2c05d006dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:39.363590 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8372b-9c32-4ffd-bb45-7c2c05d006dd None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:39.363869 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8372b-9c32-4ffd-bb45-7c2c05d006dd None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:39.364145 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8372b-9c32-4ffd-bb45-7c2c05d006dd None admin] PATH_INFO: `/v3/groups/6074835adfc045b496c12ccdf45c7672` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:39.365709 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee8372b-9c32-4ffd-bb45-7c2c05d006dd None admin] RBAC: Authorizing `identity:get_group(group_id=6074835adfc045b496c12ccdf45c7672)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:39.367614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee8372b-9c32-4ffd-bb45-7c2c05d006dd None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:39.369356 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 229/452] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Aug 1 13:53:39 2022] GET /identity/v3/groups/6074835adfc045b496c12ccdf45c7672 => generated 226 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:39.370187 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8372b-9c32-4ffd-bb45-7c2c05d006dd None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:39.378647 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7fc6d6-48df-4dc7-9cb0-7d3a0851eeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:39.379374 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fc6d6-48df-4dc7-9cb0-7d3a0851eeca None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:39.379703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fc6d6-48df-4dc7-9cb0-7d3a0851eeca None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:39.380090 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fc6d6-48df-4dc7-9cb0-7d3a0851eeca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:39.380942 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7fc6d6-48df-4dc7-9cb0-7d3a0851eeca None admin] RBAC: Adding query filter params (group.id=6074835adfc045b496c12ccdf45c7672, role.id=75aff8cb4cf748a2adbd6c3f141fe8ac, scope.project.id=7489ab03e4a8475cab57be63c14ec0c0) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:39.381259 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7fc6d6-48df-4dc7-9cb0-7d3a0851eeca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:39.382731 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7fc6d6-48df-4dc7-9cb0-7d3a0851eeca None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:39.389632 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 224/453] 10.222.0.45 () {56 vars in 1376 bytes} [Mon Aug 1 13:53:39 2022] GET /identity/v3/role_assignments?group.id=6074835adfc045b496c12ccdf45c7672&scope.project.id=7489ab03e4a8475cab57be63c14ec0c0&role.id=75aff8cb4cf748a2adbd6c3f141fe8ac => generated 611 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:39.390712 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7fc6d6-48df-4dc7-9cb0-7d3a0851eeca None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:39.589699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8372b-9c32-4ffd-bb45-7c2c05d006dd None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:39.609436 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7fc6d6-48df-4dc7-9cb0-7d3a0851eeca None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:39.628345 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:39.669323 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1862f15e-7c65-4e0c-9c70-f24c69f3162e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:39.670682 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862f15e-7c65-4e0c-9c70-f24c69f3162e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:39.671056 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862f15e-7c65-4e0c-9c70-f24c69f3162e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:39.671575 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862f15e-7c65-4e0c-9c70-f24c69f3162e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:39.674922 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1862f15e-7c65-4e0c-9c70-f24c69f3162e None admin] RBAC: Adding query filter params (group.id=a8b1953358064ba4b85be4020d28d938, role.id=3e1a9ff016c840c5b51ef7e3da9900ac, scope.project.id=2a12613c4ac44ad788edb767368b347b) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:39.674922 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1862f15e-7c65-4e0c-9c70-f24c69f3162e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:39.675699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1862f15e-7c65-4e0c-9c70-f24c69f3162e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:39.683638 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 230/454] 10.222.0.45 () {56 vars in 1376 bytes} [Mon Aug 1 13:53:39 2022] GET /identity/v3/role_assignments?group.id=a8b1953358064ba4b85be4020d28d938&scope.project.id=2a12613c4ac44ad788edb767368b347b&role.id=3e1a9ff016c840c5b51ef7e3da9900ac => generated 611 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:39.690327 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd98cf-c3e5-45fb-a52a-e6e28bd27e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:39.691340 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd98cf-c3e5-45fb-a52a-e6e28bd27e5a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:39.691848 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd98cf-c3e5-45fb-a52a-e6e28bd27e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:39.692501 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd98cf-c3e5-45fb-a52a-e6e28bd27e5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:39.694028 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd98cf-c3e5-45fb-a52a-e6e28bd27e5a None admin] RBAC: Adding query filter params (group.id=6074835adfc045b496c12ccdf45c7672, role.id=60efb70fa7354ebfb04aa5e9bdad2a78, scope.project.id=7489ab03e4a8475cab57be63c14ec0c0) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:39.695150 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd98cf-c3e5-45fb-a52a-e6e28bd27e5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:39.697628 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd98cf-c3e5-45fb-a52a-e6e28bd27e5a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:39.719380 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 225/455] 10.222.0.45 () {56 vars in 1376 bytes} [Mon Aug 1 13:53:39 2022] GET /identity/v3/role_assignments?group.id=6074835adfc045b496c12ccdf45c7672&scope.project.id=7489ab03e4a8475cab57be63c14ec0c0&role.id=60efb70fa7354ebfb04aa5e9bdad2a78 => generated 611 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:43.549202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ae672-200d-41ed-af02-9d8b5671edba None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:43.550653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ae672-200d-41ed-af02-9d8b5671edba None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:43.551323 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ae672-200d-41ed-af02-9d8b5671edba None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:43.554595 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 231/456] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:43.564312 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43404061-9734-4be9-adf3-db3468866906 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:43.565709 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43404061-9734-4be9-adf3-db3468866906 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:43.566397 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43404061-9734-4be9-adf3-db3468866906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:43.614500 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-43404061-9734-4be9-adf3-db3468866906 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:43.617056 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-43404061-9734-4be9-adf3-db3468866906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:43.676479 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:43.781058 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 226/457] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:43.812351 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6921316-d0bc-4d7d-839e-9343c9a8d38d None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:43.813287 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6921316-d0bc-4d7d-839e-9343c9a8d38d None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:43.814049 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6921316-d0bc-4d7d-839e-9343c9a8d38d None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:43.817349 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 232/458] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:43.826295 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146a2e0-e810-44d9-8633-7c3bb6109c21 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:43.827212 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146a2e0-e810-44d9-8633-7c3bb6109c21 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:43.827934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146a2e0-e810-44d9-8633-7c3bb6109c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:43.846293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6921316-d0bc-4d7d-839e-9343c9a8d38d None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:43.875809 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-3146a2e0-e810-44d9-8633-7c3bb6109c21 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:43.877849 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3146a2e0-e810-44d9-8633-7c3bb6109c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:44.050114 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 227/459] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:44.060496 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6e3d7-ee57-4021-9d4a-5784391795a3 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:44.061308 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6e3d7-ee57-4021-9d4a-5784391795a3 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:44.061992 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6e3d7-ee57-4021-9d4a-5784391795a3 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:44.065287 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 228/460] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:44 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:44.090830 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde176d4-6802-4d42-95a9-c73eaabbeaa1 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:44.091640 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde176d4-6802-4d42-95a9-c73eaabbeaa1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:44.092437 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde176d4-6802-4d42-95a9-c73eaabbeaa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:44.135405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-fde176d4-6802-4d42-95a9-c73eaabbeaa1 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:44.137798 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fde176d4-6802-4d42-95a9-c73eaabbeaa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:44.140974 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6921316-d0bc-4d7d-839e-9343c9a8d38d None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:44.238032 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29284b-6580-43fa-ac5d-0efaa2b6ca7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:44.239282 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29284b-6580-43fa-ac5d-0efaa2b6ca7e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:44.240152 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29284b-6580-43fa-ac5d-0efaa2b6ca7e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:44.241033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29284b-6580-43fa-ac5d-0efaa2b6ca7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:44.268130 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b29284b-6580-43fa-ac5d-0efaa2b6ca7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:44.271924 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b29284b-6580-43fa-ac5d-0efaa2b6ca7e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:44.288421 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 229/461] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:44.290730 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde176d4-6802-4d42-95a9-c73eaabbeaa1 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:44.312018 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-1b29284b-6580-43fa-ac5d-0efaa2b6ca7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:53:44.312018 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:44.312018 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:44.312018 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:44.312018 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:44.312018 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:44.312018 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:44.312018 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:44.312018 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:44.312018 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:44.312018 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:44.312018 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:44.312018 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:53:44.312018 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:53:44.312018 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:53:44.313769 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:53:44.313769 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:44.313769 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:44.313769 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:44.313769 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:44.313769 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:44.313769 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:44.313769 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:44.313769 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:44.313769 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:44.313769 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:44.313769 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:44.313769 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:44.313769 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:44.313769 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:44.313769 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:44.313769 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:44.315568 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:53:44.315568 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:53:44.315568 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:53:44.315568 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:44.318922 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 233/462] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:53:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 473 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:44.320923 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29284b-6580-43fa-ac5d-0efaa2b6ca7e None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:44.348310 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:44.398085 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:44.528879 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde176d4-6802-4d42-95a9-c73eaabbeaa1 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:44.562802 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29284b-6580-43fa-ac5d-0efaa2b6ca7e None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:44.615292 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa9465-c715-4033-a93a-d757a30c9c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:44.616126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa9465-c715-4033-a93a-d757a30c9c7e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:44.616126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa9465-c715-4033-a93a-d757a30c9c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:44.616693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa9465-c715-4033-a93a-d757a30c9c7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:44.631561 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa9465-c715-4033-a93a-d757a30c9c7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:44.634104 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa9465-c715-4033-a93a-d757a30c9c7e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:44.655448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-d6aa9465-c715-4033-a93a-d757a30c9c7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:53:44.655448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:44.655448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:44.655448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:44.655448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:44.655448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:44.655448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:44.655448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:44.655448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:44.655448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:44.655448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:44.655448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:44.655448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:53:44.655448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:53:44.655448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:53:44.661760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:53:44.661760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:44.661760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:44.661760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:44.661760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:44.661760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:44.661760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:44.661760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:44.661760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:44.661760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:44.661760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:44.661760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:44.661760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:44.661760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:44.661760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:44.661760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:44.661760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:44.665503 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:53:44.665503 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:53:44.665503 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:53:44.665503 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:53:44.666738 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f32da-c452-4e1f-ab33-aa24dc586a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:44.666738 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f32da-c452-4e1f-ab33-aa24dc586a71 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:44.666738 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f32da-c452-4e1f-ab33-aa24dc586a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:44.666738 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f32da-c452-4e1f-ab33-aa24dc586a71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:44.667573 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 230/463] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:53:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:44.667573 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa9465-c715-4033-a93a-d757a30c9c7e None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:44.687371 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025f32da-c452-4e1f-ab33-aa24dc586a71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:44.690288 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025f32da-c452-4e1f-ab33-aa24dc586a71 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:44.717895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-025f32da-c452-4e1f-ab33-aa24dc586a71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:53:44.717895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:44.717895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:44.717895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:44.717895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:44.717895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:44.717895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:44.717895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:44.717895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:44.717895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:44.717895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:44.717895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:44.717895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:53:44.717895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:53:44.717895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:53:44.720177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:53:44.720177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:44.720177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:44.720177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:44.720177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:44.720177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:44.720177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:44.720177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:44.720177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:44.720177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:44.720177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:44.720177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:44.720177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:44.720177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:44.720177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:44.720177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:44.720177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:44.721818 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:53:44.721818 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:53:44.721818 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:53:44.721818 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:44.730292 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 234/464] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:53:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:44.730292 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf91612-897e-4eff-a9db-01f5d3643f8b None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:44.730292 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf91612-897e-4eff-a9db-01f5d3643f8b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:44.730292 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf91612-897e-4eff-a9db-01f5d3643f8b None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:44.730292 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 235/465] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:44.731510 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf91612-897e-4eff-a9db-01f5d3643f8b None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:44.809130 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:44.864808 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa9465-c715-4033-a93a-d757a30c9c7e None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:44.929449 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7d7693-76da-463b-b921-8f829cb0c443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:44.930043 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d7693-76da-463b-b921-8f829cb0c443 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:44.930306 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d7693-76da-463b-b921-8f829cb0c443 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:44.930624 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d7693-76da-463b-b921-8f829cb0c443 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:44.931577 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7d7693-76da-463b-b921-8f829cb0c443 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:44.932158 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7d7693-76da-463b-b921-8f829cb0c443 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:44.934533 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7d7693-76da-463b-b921-8f829cb0c443 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:44.939420 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:44.957722 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 231/466] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:53:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:44.959313 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7d7693-76da-463b-b921-8f829cb0c443 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:44.987690 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf91612-897e-4eff-a9db-01f5d3643f8b None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:45.062038 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4165a25a-cefa-438e-aa3f-9a3aeb78c72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:45.063936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165a25a-cefa-438e-aa3f-9a3aeb78c72d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:45.064490 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165a25a-cefa-438e-aa3f-9a3aeb78c72d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:45.064999 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165a25a-cefa-438e-aa3f-9a3aeb78c72d None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:45.066436 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4165a25a-cefa-438e-aa3f-9a3aeb78c72d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:45.066977 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4165a25a-cefa-438e-aa3f-9a3aeb78c72d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:45.069699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4165a25a-cefa-438e-aa3f-9a3aeb78c72d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:45.100720 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 236/467] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:53:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:45.102326 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78c2ba-55b4-4d33-b0ff-7f99a770e6fb None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:45.102835 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78c2ba-55b4-4d33-b0ff-7f99a770e6fb None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:45.103325 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78c2ba-55b4-4d33-b0ff-7f99a770e6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:45.138558 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-ec78c2ba-55b4-4d33-b0ff-7f99a770e6fb None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:45.140880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ec78c2ba-55b4-4d33-b0ff-7f99a770e6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:45.181023 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7d7693-76da-463b-b921-8f829cb0c443 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:45.237339 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:45.248482 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7820e1-d8f7-4e43-9434-816cd402c9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:45.249577 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7820e1-d8f7-4e43-9434-816cd402c9f6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:45.250073 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7820e1-d8f7-4e43-9434-816cd402c9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:45.250575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7820e1-d8f7-4e43-9434-816cd402c9f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:45.251822 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7820e1-d8f7-4e43-9434-816cd402c9f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:45.252380 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7820e1-d8f7-4e43-9434-816cd402c9f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:45.254916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7820e1-d8f7-4e43-9434-816cd402c9f6 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:45.277632 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 232/468] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:53:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:45.279107 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7820e1-d8f7-4e43-9434-816cd402c9f6 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:45.282314 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 237/469] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:45.283329 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78c2ba-55b4-4d33-b0ff-7f99a770e6fb None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:45.408489 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:45.452994 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78c2ba-55b4-4d33-b0ff-7f99a770e6fb None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:45.462539 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7820e1-d8f7-4e43-9434-816cd402c9f6 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:45.510007 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057eacd8-2f00-43f5-99b5-945918838ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:45.511162 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057eacd8-2f00-43f5-99b5-945918838ca3 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:45.511446 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057eacd8-2f00-43f5-99b5-945918838ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:45.511975 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057eacd8-2f00-43f5-99b5-945918838ca3 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:45.513324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057eacd8-2f00-43f5-99b5-945918838ca3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:45.515943 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057eacd8-2f00-43f5-99b5-945918838ca3 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:45.520920 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437f071-a6e3-47c7-890a-3f6bcdaf9694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:45.521966 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437f071-a6e3-47c7-890a-3f6bcdaf9694 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:45.522409 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437f071-a6e3-47c7-890a-3f6bcdaf9694 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:45.522879 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437f071-a6e3-47c7-890a-3f6bcdaf9694 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:45.524084 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2437f071-a6e3-47c7-890a-3f6bcdaf9694 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:45.526980 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2437f071-a6e3-47c7-890a-3f6bcdaf9694 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:45.553213 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 238/470] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 13:53:45 2022] POST /identity/v3/users => generated 248 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:53:45.555821 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b7b63-55c9-4c4e-884b-cba556c87ac7 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:45.556395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b7b63-55c9-4c4e-884b-cba556c87ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:45.558898 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b7b63-55c9-4c4e-884b-cba556c87ac7 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:45.559477 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 239/471] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:45.563123 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b7b63-55c9-4c4e-884b-cba556c87ac7 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:45.575302 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 233/472] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 13:53:45 2022] POST /identity/v3/users => generated 246 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:53:45.577002 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437f071-a6e3-47c7-890a-3f6bcdaf9694 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:45.690048 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:45.770494 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b7b63-55c9-4c4e-884b-cba556c87ac7 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:45.816017 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437f071-a6e3-47c7-890a-3f6bcdaf9694 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:45.817402 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:45.828043 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54cc85d-a9d5-49ff-a810-00a9125a0947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:45.829721 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54cc85d-a9d5-49ff-a810-00a9125a0947 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:45.830188 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54cc85d-a9d5-49ff-a810-00a9125a0947 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:45.830686 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54cc85d-a9d5-49ff-a810-00a9125a0947 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:45.831613 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54cc85d-a9d5-49ff-a810-00a9125a0947 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:45.833356 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54cc85d-a9d5-49ff-a810-00a9125a0947 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:45.865954 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 240/473] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 13:53:45 2022] POST /identity/v3/users => generated 248 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:53:45.867823 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b2fe9-f0fc-45ec-8d91-eca6c41b71a5 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:45.868543 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b2fe9-f0fc-45ec-8d91-eca6c41b71a5 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:45.869179 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b2fe9-f0fc-45ec-8d91-eca6c41b71a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:45.892046 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b9697-8505-4d3e-b277-170c77f479ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:45.893692 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b9697-8505-4d3e-b277-170c77f479ca None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:45.894281 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b9697-8505-4d3e-b277-170c77f479ca None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:45.894935 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b9697-8505-4d3e-b277-170c77f479ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:45.895394 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-d31b2fe9-f0fc-45ec-8d91-eca6c41b71a5 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:45.897926 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d31b2fe9-f0fc-45ec-8d91-eca6c41b71a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:45.918847 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b9697-8505-4d3e-b277-170c77f479ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:45.922076 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b9697-8505-4d3e-b277-170c77f479ca None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:45.955294 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-e44b9697-8505-4d3e-b277-170c77f479ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:53:45.955294 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:45.955294 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:45.955294 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:45.955294 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:45.955294 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:45.955294 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:45.955294 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:45.955294 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:45.955294 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:45.955294 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:45.955294 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:45.955294 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:53:45.955294 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:53:45.955294 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:53:45.957292 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:53:45.957292 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:45.957292 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:45.957292 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:45.957292 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:45.957292 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:45.957292 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:45.957292 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:45.957292 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:45.957292 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:45.957292 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:45.957292 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:45.957292 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:45.957292 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:45.957292 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:45.957292 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:45.957292 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:45.958665 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:53:45.958665 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:53:45.958665 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:53:45.958665 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:53:45.958665 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 234/474] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:53:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:45.969982 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b9697-8505-4d3e-b277-170c77f479ca None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:46.056888 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 241/475] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:46.096918 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b2fe9-f0fc-45ec-8d91-eca6c41b71a5 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:46.180171 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:46.196007 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b9697-8505-4d3e-b277-170c77f479ca None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:46.271506 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ede0c-58eb-4800-ba7b-996ec26ccde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:46.272102 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ede0c-58eb-4800-ba7b-996ec26ccde7 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:46.272438 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ede0c-58eb-4800-ba7b-996ec26ccde7 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:46.272872 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ede0c-58eb-4800-ba7b-996ec26ccde7 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:46.274338 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78ede0c-58eb-4800-ba7b-996ec26ccde7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:46.274656 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78ede0c-58eb-4800-ba7b-996ec26ccde7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:46.276239 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78ede0c-58eb-4800-ba7b-996ec26ccde7 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:46.295519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 235/476] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:53:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:46.305908 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ede0c-58eb-4800-ba7b-996ec26ccde7 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:46.320730 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b2fe9-f0fc-45ec-8d91-eca6c41b71a5 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:46.366284 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:46.408012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f038f2-6357-4ae5-b790-1d0e7cc40647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:46.409298 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f038f2-6357-4ae5-b790-1d0e7cc40647 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:46.409800 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f038f2-6357-4ae5-b790-1d0e7cc40647 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:46.410270 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f038f2-6357-4ae5-b790-1d0e7cc40647 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:46.411594 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f038f2-6357-4ae5-b790-1d0e7cc40647 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:46.413949 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f038f2-6357-4ae5-b790-1d0e7cc40647 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:46.428878 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.sql.core [None req-e9f038f2-6357-4ae5-b790-1d0e7cc40647 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 13:53:46.428878 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 01 13:53:46.428878 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [parameters: {'id': 'b81307063bb144238ff5035579a2fd7e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 01 13:53:46.428878 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80310) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 13:53:46.460091 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ede0c-58eb-4800-ba7b-996ec26ccde7 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:46.464849 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-e9f038f2-6357-4ae5-b790-1d0e7cc40647 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 01 13:53:46.464849 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:46.464849 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 13:53:46.464849 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 13:53:46.464849 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 13:53:46.464849 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 13:53:46.464849 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 13:53:46.464849 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 13:53:46.464849 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 13:53:46.464849 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application conn.query(q) Aug 01 13:53:46.464849 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 13:53:46.464849 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 13:53:46.464849 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 13:53:46.464849 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application result.read() Aug 01 13:53:46.464849 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 13:53:46.466457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 13:53:46.466457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 13:53:46.466457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 13:53:46.466457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 13:53:46.466457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 13:53:46.466457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 13:53:46.466457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 13:53:46.466457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 13:53:46.466457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:46.466457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 13:53:46.466457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:46.466457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:46.466457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 13:53:46.466457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 13:53:46.466457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 01 13:53:46.466457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return ref.to_dict() Aug 01 13:53:46.466457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 13:53:46.466457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application next(self.gen) Aug 01 13:53:46.468218 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 13:53:46.468218 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application yield resource Aug 01 13:53:46.468218 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 13:53:46.468218 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application next(self.gen) Aug 01 13:53:46.468218 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 13:53:46.468218 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 13:53:46.468218 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 13:53:46.468218 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 13:53:46.468218 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 13:53:46.468218 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise self.value Aug 01 13:53:46.468218 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 13:53:46.468218 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 13:53:46.468218 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 13:53:46.468218 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application session.commit() Aug 01 13:53:46.468218 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 13:53:46.468218 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 13:53:46.468218 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 13:53:46.470747 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 13:53:46.470747 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 13:53:46.470747 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self.session.flush() Aug 01 13:53:46.470747 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 13:53:46.470747 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 13:53:46.470747 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 13:53:46.470747 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 13:53:46.470747 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 13:53:46.470747 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application compat.raise_( Aug 01 13:53:46.470747 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 13:53:46.470747 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception Aug 01 13:53:46.470747 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 13:53:46.470747 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 13:53:46.470747 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 13:53:46.470747 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 13:53:46.470747 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 13:53:46.470747 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 13:53:46.472812 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 13:53:46.472812 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 13:53:46.472812 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 01 13:53:46.472812 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 01 13:53:46.472812 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 13:53:46.472812 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 13:53:46.472812 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 13:53:46.472812 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 13:53:46.472812 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 13:53:46.472812 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 13:53:46.472812 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 13:53:46.472812 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 13:53:46.472812 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 13:53:46.472812 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 13:53:46.472812 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 13:53:46.472812 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception Aug 01 13:53:46.474884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 13:53:46.474884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 13:53:46.474884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 13:53:46.474884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 13:53:46.474884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 13:53:46.474884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 13:53:46.474884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 13:53:46.474884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application conn.query(q) Aug 01 13:53:46.474884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 13:53:46.474884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 13:53:46.474884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 13:53:46.474884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application result.read() Aug 01 13:53:46.474884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 13:53:46.474884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 13:53:46.474884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 13:53:46.474884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 13:53:46.481735 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 13:53:46.481735 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 13:53:46.481735 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 13:53:46.481735 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 13:53:46.481735 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 13:53:46.481735 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 01 13:53:46.481735 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [parameters: {'id': 'b81307063bb144238ff5035579a2fd7e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 01 13:53:46.481735 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 13:53:46.481735 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:46.481735 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:53:46.481735 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:46.481735 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:46.481735 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:46.481735 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:46.481735 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:46.481735 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:46.483377 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:46.483377 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:46.483377 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:46.483377 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:46.483377 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:46.483377 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:46.483377 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 01 13:53:46.483377 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 01 13:53:46.483377 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:46.483377 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:46.483377 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 01 13:53:46.483377 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 01 13:53:46.483377 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 13:53:46.483377 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 13:53:46.483377 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 01 13:53:46.483377 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:46.485333 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 242/477] 10.222.0.45 () {60 vars in 1131 bytes} [Mon Aug 1 13:53:46 2022] POST /identity/v3/roles => generated 144 bytes in 373 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:53:46.494261 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f038f2-6357-4ae5-b790-1d0e7cc40647 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:46.511022 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96622d-0da6-4e94-b9c6-2fe0601ab790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:46.511671 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96622d-0da6-4e94-b9c6-2fe0601ab790 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:46.511956 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96622d-0da6-4e94-b9c6-2fe0601ab790 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:46.512779 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96622d-0da6-4e94-b9c6-2fe0601ab790 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:46.514388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce96622d-0da6-4e94-b9c6-2fe0601ab790 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:46.517403 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce96622d-0da6-4e94-b9c6-2fe0601ab790 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:46.556390 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 236/478] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 13:53:46 2022] POST /identity/v3/users => generated 249 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 13:53:46.721244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f038f2-6357-4ae5-b790-1d0e7cc40647 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:46.814395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca8fd0-ad99-483f-ba75-ceee3223d206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:46.816182 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca8fd0-ad99-483f-ba75-ceee3223d206 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:46.817205 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca8fd0-ad99-483f-ba75-ceee3223d206 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:46.818196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca8fd0-ad99-483f-ba75-ceee3223d206 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:46.853155 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca8fd0-ad99-483f-ba75-ceee3223d206 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:46.866677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-15ca8fd0-ad99-483f-ba75-ceee3223d206 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:53:46.866677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:46.866677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:46.866677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:46.866677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:46.866677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:46.866677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:46.866677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:46.866677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:46.866677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:46.866677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:46.866677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:46.866677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:46.866677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:46.866677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:46.866677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:46.868431 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:46.868431 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:46.868431 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:46.868431 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:46.868431 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:46.868431 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:46.868431 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:46.868431 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:46.868431 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:46.868431 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:46.868431 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:46.868431 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:46.868431 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:46.868431 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:46.868431 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:46.868431 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:46.869951 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:46.869951 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:46.869951 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:46.869951 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:46.869951 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:46.869951 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:46.869951 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:46.869951 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:46.869951 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:46.869951 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:46.869951 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:53:46.869951 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:46.874103 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 243/479] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 13:53:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:46.883421 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96622d-0da6-4e94-b9c6-2fe0601ab790 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:46.919880 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:47.066012 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96622d-0da6-4e94-b9c6-2fe0601ab790 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:47.116883 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a749fbe-3c37-4f8f-9c08-d17350973e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:47.117608 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a749fbe-3c37-4f8f-9c08-d17350973e9e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:47.117931 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a749fbe-3c37-4f8f-9c08-d17350973e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:47.118287 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a749fbe-3c37-4f8f-9c08-d17350973e9e None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:47.119394 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a749fbe-3c37-4f8f-9c08-d17350973e9e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:47.119859 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a749fbe-3c37-4f8f-9c08-d17350973e9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:47.121743 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a749fbe-3c37-4f8f-9c08-d17350973e9e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:47.138355 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 237/480] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:53:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:49.523285 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958b7f2-c9a4-41ba-be8b-440ff1106500 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:49.524079 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958b7f2-c9a4-41ba-be8b-440ff1106500 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:49.524541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958b7f2-c9a4-41ba-be8b-440ff1106500 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:49.527404 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 244/481] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:49.537509 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f1a2f-6b54-476d-b1d9-815c0cdf0634 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:49.538371 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f1a2f-6b54-476d-b1d9-815c0cdf0634 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:49.538508 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f1a2f-6b54-476d-b1d9-815c0cdf0634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:49.584461 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-3e4f1a2f-6b54-476d-b1d9-815c0cdf0634 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:49.587076 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3e4f1a2f-6b54-476d-b1d9-815c0cdf0634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:49.737360 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 238/482] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:49.777440 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958b7f2-c9a4-41ba-be8b-440ff1106500 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:49.790605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa053e-93fc-42be-92eb-a12e57fd2e2e None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:49.791523 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa053e-93fc-42be-92eb-a12e57fd2e2e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:49.792231 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa053e-93fc-42be-92eb-a12e57fd2e2e None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:49.795329 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 239/483] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:49.819880 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27805c-8f51-4761-996c-840d321ced9e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:49.820226 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27805c-8f51-4761-996c-840d321ced9e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:49.820548 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27805c-8f51-4761-996c-840d321ced9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:49.848814 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:49.869771 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:49.870217 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-2c27805c-8f51-4761-996c-840d321ced9e None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:49.872210 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2c27805c-8f51-4761-996c-840d321ced9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:50.078730 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958b7f2-c9a4-41ba-be8b-440ff1106500 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:50.114718 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 240/484] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:53:50.165851 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c27805c-8f51-4761-996c-840d321ced9e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:50.170876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be2b399-853b-439d-94cd-cfdc4dc5cbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:50.172028 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2b399-853b-439d-94cd-cfdc4dc5cbea None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:50.172562 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2b399-853b-439d-94cd-cfdc4dc5cbea None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:50.173034 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2b399-853b-439d-94cd-cfdc4dc5cbea None admin] PATH_INFO: `/v3/roles/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:50.198740 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be2b399-853b-439d-94cd-cfdc4dc5cbea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:50.209624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-6be2b399-853b-439d-94cd-cfdc4dc5cbea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:53:50.209624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:50.209624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:50.209624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:50.209624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:50.209624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:50.209624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:50.209624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:50.209624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:50.209624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:50.209624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:50.209624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:50.209624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:50.209624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:50.209624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:50.209624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:50.210624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:50.210624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:50.210624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:50.210624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:50.210624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:50.210624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:50.210624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:50.210624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:50.210624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:50.210624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:50.210624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:50.210624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:50.210624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:50.210624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:50.210624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:50.210624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:50.211578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:50.211578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:50.211578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:50.211578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:50.211578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:50.211578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:50.211578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:50.211578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:50.211578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:50.211578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:50.211578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:53:50.211578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:50.212680 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 245/485] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 13:53:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 436 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:50.214435 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c80cb2-0318-4585-91a2-3e864a90e9bf None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:50.214943 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c80cb2-0318-4585-91a2-3e864a90e9bf None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:50.215421 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c80cb2-0318-4585-91a2-3e864a90e9bf None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:50.223493 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 246/486] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:50.225063 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c80cb2-0318-4585-91a2-3e864a90e9bf None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:50.344891 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c27805c-8f51-4761-996c-840d321ced9e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:50.410713 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92700d52-51f7-403e-a889-dda44874d61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:50.412181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92700d52-51f7-403e-a889-dda44874d61c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:50.412674 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92700d52-51f7-403e-a889-dda44874d61c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:50.413124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92700d52-51f7-403e-a889-dda44874d61c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:50.422561 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:50.439919 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92700d52-51f7-403e-a889-dda44874d61c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:50.449998 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-92700d52-51f7-403e-a889-dda44874d61c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:53:50.449998 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:50.449998 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:50.449998 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:50.449998 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:50.449998 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:50.449998 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:50.449998 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:50.449998 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:50.449998 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:50.449998 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:50.449998 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:50.449998 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:50.449998 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:50.449998 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:50.449998 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:50.451993 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:50.451993 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:50.451993 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:50.451993 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:50.451993 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:50.451993 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:50.451993 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:50.451993 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:50.451993 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:50.451993 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:50.451993 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:50.451993 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:50.451993 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:50.451993 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:50.451993 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:50.451993 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:50.454022 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:50.454022 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:50.454022 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:50.454022 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:50.454022 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:50.454022 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:50.454022 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:50.454022 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:50.454022 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:50.454022 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:50.454022 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:53:50.454022 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:53:50.454022 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 241/487] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 13:53:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:50.456026 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d857671-be3d-4e8b-98ef-d0835ac3a7cc None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:50.456026 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d857671-be3d-4e8b-98ef-d0835ac3a7cc None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:50.456580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d857671-be3d-4e8b-98ef-d0835ac3a7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:50.459665 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c80cb2-0318-4585-91a2-3e864a90e9bf None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:50.463487 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:50.497979 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-3d857671-be3d-4e8b-98ef-d0835ac3a7cc None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:50.502039 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3d857671-be3d-4e8b-98ef-d0835ac3a7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:50.531864 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784cc4a-959b-4ea2-a4f1-04acb1bcc08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:50.533190 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784cc4a-959b-4ea2-a4f1-04acb1bcc08d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:50.533448 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784cc4a-959b-4ea2-a4f1-04acb1bcc08d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:50.533887 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784cc4a-959b-4ea2-a4f1-04acb1bcc08d None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:50.535226 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7784cc4a-959b-4ea2-a4f1-04acb1bcc08d None admin] RBAC: Adding query filter params (name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:50.535735 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7784cc4a-959b-4ea2-a4f1-04acb1bcc08d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:50.538528 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7784cc4a-959b-4ea2-a4f1-04acb1bcc08d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:50.556920 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 247/488] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 13:53:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:50.560625 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301f63e-d1f5-4f2f-92b4-2883a3f354e8 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:50.561326 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301f63e-d1f5-4f2f-92b4-2883a3f354e8 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:50.561817 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301f63e-d1f5-4f2f-92b4-2883a3f354e8 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:50.564793 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 248/489] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:50.567229 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0301f63e-d1f5-4f2f-92b4-2883a3f354e8 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:50.659479 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 242/490] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:50.660482 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d857671-be3d-4e8b-98ef-d0835ac3a7cc None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:50.775507 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0301f63e-d1f5-4f2f-92b4-2883a3f354e8 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:50.847823 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b40bd-e51a-42b1-9f70-d66dd8c919d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:50.848850 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b40bd-e51a-42b1-9f70-d66dd8c919d2 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:50.849489 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b40bd-e51a-42b1-9f70-d66dd8c919d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:50.849804 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b40bd-e51a-42b1-9f70-d66dd8c919d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:50.850994 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b40bd-e51a-42b1-9f70-d66dd8c919d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:50.851407 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b40bd-e51a-42b1-9f70-d66dd8c919d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:50.853074 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b40bd-e51a-42b1-9f70-d66dd8c919d2 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:50.876626 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 249/491] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 13:53:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:50.876626 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03cefdc-6632-486d-9abe-1a96da92885d None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:50.876626 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03cefdc-6632-486d-9abe-1a96da92885d None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:50.876626 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03cefdc-6632-486d-9abe-1a96da92885d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:50.901537 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-f03cefdc-6632-486d-9abe-1a96da92885d None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:50.903146 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-f03cefdc-6632-486d-9abe-1a96da92885d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:50.903837 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d857671-be3d-4e8b-98ef-d0835ac3a7cc None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:50.939978 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:50.967189 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:50.999379 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25eabcf-adaf-4e21-84e5-41273df9fb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:51.000605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25eabcf-adaf-4e21-84e5-41273df9fb4a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:51.001085 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25eabcf-adaf-4e21-84e5-41273df9fb4a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:51.001504 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25eabcf-adaf-4e21-84e5-41273df9fb4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:51.021415 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25eabcf-adaf-4e21-84e5-41273df9fb4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:51.023701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25eabcf-adaf-4e21-84e5-41273df9fb4a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:51.048197 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-a25eabcf-adaf-4e21-84e5-41273df9fb4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:53:51.048197 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:51.048197 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:51.048197 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:51.048197 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:51.048197 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:51.048197 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:51.048197 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:51.048197 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:51.048197 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:51.048197 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:51.048197 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:51.048197 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:53:51.048197 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:53:51.048197 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:53:51.049623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:53:51.049623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:51.049623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:51.049623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:51.049623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:51.049623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:51.049623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:51.049623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:51.049623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:51.049623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:51.049623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:51.049623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:51.049623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:51.049623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:51.049623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:51.049623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:51.049623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:51.050967 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:53:51.050967 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:53:51.050967 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:53:51.050967 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:53:51.052145 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 243/492] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:53:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:51.052338 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 250/493] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:51.052662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25eabcf-adaf-4e21-84e5-41273df9fb4a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:51.053616 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03cefdc-6632-486d-9abe-1a96da92885d None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:51.230097 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25eabcf-adaf-4e21-84e5-41273df9fb4a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:51.233007 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03cefdc-6632-486d-9abe-1a96da92885d None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:51.298018 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef6f29-4738-4d51-a9a7-438f8fc89779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:51.299033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef6f29-4738-4d51-a9a7-438f8fc89779 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:51.299477 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef6f29-4738-4d51-a9a7-438f8fc89779 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:51.299904 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef6f29-4738-4d51-a9a7-438f8fc89779 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:51.303219 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b895f76-f3bb-458f-be2d-3e0f5565c1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:51.304415 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b895f76-f3bb-458f-be2d-3e0f5565c1a1 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:51.304920 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b895f76-f3bb-458f-be2d-3e0f5565c1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:51.305430 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b895f76-f3bb-458f-be2d-3e0f5565c1a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:51.318514 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef6f29-4738-4d51-a9a7-438f8fc89779 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:51.320896 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef6f29-4738-4d51-a9a7-438f8fc89779 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:51.325598 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b895f76-f3bb-458f-be2d-3e0f5565c1a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:51.334815 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-1b895f76-f3bb-458f-be2d-3e0f5565c1a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:53:51.334815 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:51.334815 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:51.334815 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:51.334815 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:51.334815 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:51.334815 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:51.334815 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:51.334815 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:51.334815 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:51.334815 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:51.334815 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:51.334815 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:51.334815 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:51.334815 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:51.334815 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:51.336864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:51.336864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:51.336864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:51.336864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:51.336864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:51.336864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:51.336864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:51.336864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:51.336864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:51.336864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:51.336864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:51.336864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:51.336864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:51.336864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:51.336864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:51.336864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:51.339060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:51.339060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:51.339060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:51.339060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:51.339060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:51.339060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:51.339060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:51.339060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:51.339060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:51.339060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:51.339060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:53:51.339060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:53:51.339060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 244/494] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 13:53:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:51.339060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576849e-eab0-4743-b7fa-6585f3fde17a None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:51.340558 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576849e-eab0-4743-b7fa-6585f3fde17a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:51.340558 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576849e-eab0-4743-b7fa-6585f3fde17a None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:51.341589 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 245/495] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:51.342712 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576849e-eab0-4743-b7fa-6585f3fde17a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:51.354677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-bdef6f29-4738-4d51-a9a7-438f8fc89779 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:53:51.354677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:51.354677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:51.354677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:51.354677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:51.354677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:51.354677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:51.354677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:51.354677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:51.354677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:51.354677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:51.354677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:51.354677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:53:51.354677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:53:51.354677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:53:51.356172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:53:51.356172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:51.356172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:51.356172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:51.356172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:51.356172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:51.356172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:51.356172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:51.356172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:51.356172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:51.356172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:51.356172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:51.356172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:51.356172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:51.356172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:51.356172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:51.356172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:51.357646 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:53:51.357646 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:53:51.357646 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:53:51.357646 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:51.358828 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 251/496] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:53:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:51.360754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef6f29-4738-4d51-a9a7-438f8fc89779 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:51.388612 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:51.434199 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:51.566521 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef6f29-4738-4d51-a9a7-438f8fc89779 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:51.571925 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576849e-eab0-4743-b7fa-6585f3fde17a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:51.622711 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812362a9-505d-41e1-92f9-d8ec84fbb13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:51.623898 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812362a9-505d-41e1-92f9-d8ec84fbb13f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:51.624496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812362a9-505d-41e1-92f9-d8ec84fbb13f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:51.625064 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812362a9-505d-41e1-92f9-d8ec84fbb13f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:51.631508 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7605f4-d0a2-47d5-a80a-e533623d53e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:51.632132 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7605f4-d0a2-47d5-a80a-e533623d53e1 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:51.633897 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7605f4-d0a2-47d5-a80a-e533623d53e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:51.634201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7605f4-d0a2-47d5-a80a-e533623d53e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:51.634936 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7605f4-d0a2-47d5-a80a-e533623d53e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:51.635247 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7605f4-d0a2-47d5-a80a-e533623d53e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:51.639562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7605f4-d0a2-47d5-a80a-e533623d53e1 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:51.640388 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812362a9-505d-41e1-92f9-d8ec84fbb13f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:51.649800 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-812362a9-505d-41e1-92f9-d8ec84fbb13f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:53:51.649800 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:51.649800 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:51.649800 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:51.649800 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:51.649800 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:51.649800 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:51.649800 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:51.649800 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:51.649800 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:51.649800 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:51.649800 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:51.649800 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:51.649800 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:51.649800 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:51.649800 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:51.651791 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:51.651791 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:51.651791 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:51.651791 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:51.651791 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:51.651791 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:51.651791 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:51.651791 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:51.651791 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:51.651791 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:51.651791 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:51.651791 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:51.651791 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:51.651791 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:51.651791 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:51.651791 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:51.653365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:51.653365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:51.653365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:51.653365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:51.653365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:51.653365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:51.653365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:51.653365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:51.653365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:51.653365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:51.653365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:53:51.653365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:51.653365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 252/497] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 13:53:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:51.659950 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 246/498] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:53:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:51.661292 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d1478-a347-443c-a4a4-e94dbce316f3 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:51.661722 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d1478-a347-443c-a4a4-e94dbce316f3 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:51.662225 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d1478-a347-443c-a4a4-e94dbce316f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:51.666802 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812362a9-505d-41e1-92f9-d8ec84fbb13f None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:51.687125 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-5d6d1478-a347-443c-a4a4-e94dbce316f3 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:51.689113 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5d6d1478-a347-443c-a4a4-e94dbce316f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:51.808398 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 247/499] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:51.809737 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d1478-a347-443c-a4a4-e94dbce316f3 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:51.829641 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:51.859080 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:51.898308 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812362a9-505d-41e1-92f9-d8ec84fbb13f None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:51.960571 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a6d27-2ebc-4257-97b9-ac496f008455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:51.961477 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a6d27-2ebc-4257-97b9-ac496f008455 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:51.961957 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a6d27-2ebc-4257-97b9-ac496f008455 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:51.962410 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a6d27-2ebc-4257-97b9-ac496f008455 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:51.963688 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a6d27-2ebc-4257-97b9-ac496f008455 None admin] RBAC: Adding query filter params (name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:51.964181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a6d27-2ebc-4257-97b9-ac496f008455 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:51.966851 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a6d27-2ebc-4257-97b9-ac496f008455 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:51.982118 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 253/500] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 13:53:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:51.983231 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a6d27-2ebc-4257-97b9-ac496f008455 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:51.994947 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d1478-a347-443c-a4a4-e94dbce316f3 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:52.054077 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f365a-f577-426a-8669-ea2ce783e6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:52.055092 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f365a-f577-426a-8669-ea2ce783e6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:52.055570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f365a-f577-426a-8669-ea2ce783e6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:52.055805 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f365a-f577-426a-8669-ea2ce783e6c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:52.056621 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f365a-f577-426a-8669-ea2ce783e6c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:52.056926 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f365a-f577-426a-8669-ea2ce783e6c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:52.058351 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f365a-f577-426a-8669-ea2ce783e6c7 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:52.076005 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 248/501] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:53:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:52.077188 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f365a-f577-426a-8669-ea2ce783e6c7 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:52.173136 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a6d27-2ebc-4257-97b9-ac496f008455 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:52.231302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:52.251667 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:52.255565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf433a55-8971-4969-94c9-74ae3d9d42da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:52.256773 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf433a55-8971-4969-94c9-74ae3d9d42da None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:52.257273 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf433a55-8971-4969-94c9-74ae3d9d42da None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:52.257793 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf433a55-8971-4969-94c9-74ae3d9d42da None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:52.259467 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf433a55-8971-4969-94c9-74ae3d9d42da None admin] RBAC: Adding query filter params (name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:52.260033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf433a55-8971-4969-94c9-74ae3d9d42da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:52.263418 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf433a55-8971-4969-94c9-74ae3d9d42da None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:52.273954 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f365a-f577-426a-8669-ea2ce783e6c7 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:52.279792 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 254/502] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 13:53:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:52.281422 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf433a55-8971-4969-94c9-74ae3d9d42da None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:52.351142 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77983a6-da45-4eb7-bbde-ba487034f0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:52.352174 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77983a6-da45-4eb7-bbde-ba487034f0cc None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:52.352702 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77983a6-da45-4eb7-bbde-ba487034f0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:52.353177 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77983a6-da45-4eb7-bbde-ba487034f0cc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:52.368654 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77983a6-da45-4eb7-bbde-ba487034f0cc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:52.371582 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77983a6-da45-4eb7-bbde-ba487034f0cc None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:52.397631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-b77983a6-da45-4eb7-bbde-ba487034f0cc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:53:52.397631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:52.397631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:52.397631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:52.397631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:52.397631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:52.397631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:52.397631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:52.397631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:52.397631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:52.397631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:52.397631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:52.397631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:53:52.397631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:53:52.397631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:53:52.397631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:53:52.399405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:52.399405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:52.399405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:53:52.399405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:53:52.399405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:53:52.399405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:53:52.399405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:52.399405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:52.399405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:52.399405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:52.399405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:52.399405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:52.399405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:52.399405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:52.399405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:52.399405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:52.399405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:52.399405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:52.401126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:52.401126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:52.401126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:53:52.401126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:53:52.401126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:53:52.401126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:53:52.401126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:53:52.401126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:53:52.401126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:53:52.401126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:53:52.401126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 249/503] 10.222.0.45 () {56 vars in 1093 bytes} [Mon Aug 1 13:53:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:52.402152 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77983a6-da45-4eb7-bbde-ba487034f0cc None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:52.476863 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf433a55-8971-4969-94c9-74ae3d9d42da None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:52.544740 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90bc57-531c-4e28-937f-229c54c2b59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:52.545901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90bc57-531c-4e28-937f-229c54c2b59d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:52.546446 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90bc57-531c-4e28-937f-229c54c2b59d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:52.547020 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90bc57-531c-4e28-937f-229c54c2b59d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:52.573364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e90bc57-531c-4e28-937f-229c54c2b59d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:52.576154 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e90bc57-531c-4e28-937f-229c54c2b59d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:52.604083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-9e90bc57-531c-4e28-937f-229c54c2b59d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:53:52.604083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:52.604083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:52.604083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:52.604083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:52.604083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:52.604083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:52.604083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:52.604083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:52.604083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:52.604083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:52.604083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:52.604083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:53:52.604083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:53:52.604083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:53:52.605614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:53:52.605614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:52.605614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:52.605614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:52.605614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:52.605614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:52.605614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:52.605614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:52.605614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:52.605614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:52.605614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:52.605614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:52.605614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:52.605614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:52.605614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:52.605614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:52.605614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:52.607087 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:53:52.607087 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:53:52.607087 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:53:52.607087 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:52.607087 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 255/504] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:53:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:52.607552 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77983a6-da45-4eb7-bbde-ba487034f0cc None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:52.609174 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90bc57-531c-4e28-937f-229c54c2b59d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:52.671855 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcab5417-5859-47a6-a50e-812104368ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:52.673010 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab5417-5859-47a6-a50e-812104368ba8 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:52.673504 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab5417-5859-47a6-a50e-812104368ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:52.673997 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab5417-5859-47a6-a50e-812104368ba8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:52.687015 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:52.694927 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcab5417-5859-47a6-a50e-812104368ba8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:52.697874 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcab5417-5859-47a6-a50e-812104368ba8 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:52.714366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:52.728147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-dcab5417-5859-47a6-a50e-812104368ba8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:53:52.728147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:52.728147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:52.728147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:52.728147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:52.728147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:52.728147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:52.728147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:52.728147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:52.728147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:52.728147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:52.728147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:52.728147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:53:52.728147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:53:52.728147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:53:52.729870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:53:52.729870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:52.729870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:52.729870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:52.729870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:52.729870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:52.729870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:52.729870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:52.729870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:52.729870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:52.729870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:52.729870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:52.729870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:52.729870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:52.729870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:52.729870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:52.729870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:52.731514 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:53:52.731514 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:53:52.731514 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:53:52.731514 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:53:52.731514 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 250/505] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:53:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:52.732470 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcab5417-5859-47a6-a50e-812104368ba8 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:52.768329 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90bc57-531c-4e28-937f-229c54c2b59d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:52.815335 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79390cf-f00b-4cf1-9129-08441c9256e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:52.816124 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79390cf-f00b-4cf1-9129-08441c9256e5 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:52.816622 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79390cf-f00b-4cf1-9129-08441c9256e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:52.817051 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79390cf-f00b-4cf1-9129-08441c9256e5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:52.830836 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79390cf-f00b-4cf1-9129-08441c9256e5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:52.833158 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79390cf-f00b-4cf1-9129-08441c9256e5 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:52.856009 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-e79390cf-f00b-4cf1-9129-08441c9256e5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 13:53:52.856009 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:52.856009 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:52.856009 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:52.856009 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:52.856009 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:52.856009 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:52.856009 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:52.856009 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:52.856009 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:52.856009 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:52.856009 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:52.856009 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:53:52.856009 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:53:52.856009 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:53:52.856009 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:53:52.857780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:52.857780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:52.857780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:53:52.857780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:53:52.857780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:53:52.857780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:53:52.857780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:52.857780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:52.857780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:52.857780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:52.857780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:52.857780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:52.857780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:52.857780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:52.857780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:52.857780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:52.857780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:52.857780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:52.860597 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:52.860597 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:52.860597 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:53:52.860597 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:53:52.860597 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:53:52.860597 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:53:52.860597 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:53:52.860597 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:53:52.860597 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 13:53:52.860597 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:52.860597 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 256/506] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:53:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:52.860597 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79390cf-f00b-4cf1-9129-08441c9256e5 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:52.897526 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcab5417-5859-47a6-a50e-812104368ba8 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:52.962337 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eedcdb-31d8-41fe-a75d-8f7fc3079477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:52.963226 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eedcdb-31d8-41fe-a75d-8f7fc3079477 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:52.963615 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eedcdb-31d8-41fe-a75d-8f7fc3079477 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:52.963997 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eedcdb-31d8-41fe-a75d-8f7fc3079477 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:52.981941 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eedcdb-31d8-41fe-a75d-8f7fc3079477 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:52.984359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eedcdb-31d8-41fe-a75d-8f7fc3079477 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:53.005575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-15eedcdb-31d8-41fe-a75d-8f7fc3079477 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:53:53.005575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:53.005575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:53.005575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:53.005575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:53.005575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:53.005575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:53.005575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:53.005575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:53.005575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:53.005575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:53.005575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:53.005575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:53:53.005575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:53:53.005575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:53:53.007263 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:53:53.007263 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:53.007263 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:53.007263 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:53.007263 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:53.007263 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:53.007263 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:53.007263 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:53.007263 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:53.007263 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:53.007263 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:53.007263 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:53.007263 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:53.007263 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:53.007263 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:53.007263 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:53.007263 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:53.008930 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:53:53.008930 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:53:53.008930 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:53:53.008930 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:53:53.008930 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 251/507] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:53:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:53.008930 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eedcdb-31d8-41fe-a75d-8f7fc3079477 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:53.055467 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79390cf-f00b-4cf1-9129-08441c9256e5 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:53.062329 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:53.083448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:53.127655 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b89f7-07ea-4cfd-96f1-1866ed6b6984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:53.128686 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b89f7-07ea-4cfd-96f1-1866ed6b6984 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:53.129161 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b89f7-07ea-4cfd-96f1-1866ed6b6984 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:53.129639 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b89f7-07ea-4cfd-96f1-1866ed6b6984 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:53.131258 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b89f7-07ea-4cfd-96f1-1866ed6b6984 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:53.131801 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b89f7-07ea-4cfd-96f1-1866ed6b6984 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:53.134704 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b89f7-07ea-4cfd-96f1-1866ed6b6984 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:53.160964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eedcdb-31d8-41fe-a75d-8f7fc3079477 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:53.212247 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 257/508] 10.222.0.45 () {56 vars in 1133 bytes} [Mon Aug 1 13:53:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:53.213867 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b89f7-07ea-4cfd-96f1-1866ed6b6984 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:53.227963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc8323-c684-49bb-b08e-ef5d5c2d5eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:53.228933 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc8323-c684-49bb-b08e-ef5d5c2d5eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:53.229354 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc8323-c684-49bb-b08e-ef5d5c2d5eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:53.229763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc8323-c684-49bb-b08e-ef5d5c2d5eb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:53.230856 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc8323-c684-49bb-b08e-ef5d5c2d5eb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:53.231321 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc8323-c684-49bb-b08e-ef5d5c2d5eb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:53.233652 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc8323-c684-49bb-b08e-ef5d5c2d5eb7 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:53.254611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 252/509] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:53:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:53.255679 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc8323-c684-49bb-b08e-ef5d5c2d5eb7 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:53.415231 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b89f7-07ea-4cfd-96f1-1866ed6b6984 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:53.447847 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc8323-c684-49bb-b08e-ef5d5c2d5eb7 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:53.464413 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:53.490125 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318355df-b9d8-4fdc-83fa-4f272e4fd40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:53.491242 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318355df-b9d8-4fdc-83fa-4f272e4fd40a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:53.491739 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318355df-b9d8-4fdc-83fa-4f272e4fd40a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:53.492243 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318355df-b9d8-4fdc-83fa-4f272e4fd40a None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:53.493578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318355df-b9d8-4fdc-83fa-4f272e4fd40a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:53.494119 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318355df-b9d8-4fdc-83fa-4f272e4fd40a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:53.496828 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318355df-b9d8-4fdc-83fa-4f272e4fd40a None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:53.498764 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1f6cdc-913e-44fd-9b2b-7449dcbc5915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:53.499864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f6cdc-913e-44fd-9b2b-7449dcbc5915 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:53.500382 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f6cdc-913e-44fd-9b2b-7449dcbc5915 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:53.500836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f6cdc-913e-44fd-9b2b-7449dcbc5915 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:53.502237 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1f6cdc-913e-44fd-9b2b-7449dcbc5915 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:53.502747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1f6cdc-913e-44fd-9b2b-7449dcbc5915 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:53.505921 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1f6cdc-913e-44fd-9b2b-7449dcbc5915 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:53.527272 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 258/510] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:53:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:53.528786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318355df-b9d8-4fdc-83fa-4f272e4fd40a None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:53.549990 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 253/511] 10.222.0.45 () {56 vars in 1137 bytes} [Mon Aug 1 13:53:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:53.550891 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1f6cdc-913e-44fd-9b2b-7449dcbc5915 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:53.563422 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:53.655622 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318355df-b9d8-4fdc-83fa-4f272e4fd40a None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:53.694494 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88273942-4fab-4785-b9f7-3eec2440e875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:53.695222 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88273942-4fab-4785-b9f7-3eec2440e875 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:53.695590 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88273942-4fab-4785-b9f7-3eec2440e875 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:53.695948 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88273942-4fab-4785-b9f7-3eec2440e875 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:53.696884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88273942-4fab-4785-b9f7-3eec2440e875 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:53.697401 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88273942-4fab-4785-b9f7-3eec2440e875 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:53.698966 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88273942-4fab-4785-b9f7-3eec2440e875 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:53.708973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1f6cdc-913e-44fd-9b2b-7449dcbc5915 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:53.713969 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 259/512] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:53:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:53.715047 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88273942-4fab-4785-b9f7-3eec2440e875 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:53.770692 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da7081-a327-4dae-aa1f-5f33a8c3c271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:53.771427 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da7081-a327-4dae-aa1f-5f33a8c3c271 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:53.771756 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da7081-a327-4dae-aa1f-5f33a8c3c271 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:53.772074 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da7081-a327-4dae-aa1f-5f33a8c3c271 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:53.822684 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:53.860933 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da7081-a327-4dae-aa1f-5f33a8c3c271 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:53.862458 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da7081-a327-4dae-aa1f-5f33a8c3c271 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:53.866003 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:53.901402 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88273942-4fab-4785-b9f7-3eec2440e875 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:53.964339 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f93a132-4958-4a2b-872f-602c2c274bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:53.964903 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93a132-4958-4a2b-872f-602c2c274bd1 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:53.965221 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93a132-4958-4a2b-872f-602c2c274bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:53.965542 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93a132-4958-4a2b-872f-602c2c274bd1 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:53.966260 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f93a132-4958-4a2b-872f-602c2c274bd1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:53.967808 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f93a132-4958-4a2b-872f-602c2c274bd1 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:53.994398 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 260/513] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 13:53:53 2022] POST /identity/v3/users => generated 247 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:53:53.996966 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f93a132-4958-4a2b-872f-602c2c274bd1 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:54.057284 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 254/514] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:53:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:53:54.059057 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da7081-a327-4dae-aa1f-5f33a8c3c271 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:54.215966 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:54.226569 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f93a132-4958-4a2b-872f-602c2c274bd1 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:54.275321 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da7081-a327-4dae-aa1f-5f33a8c3c271 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:54.278255 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55b04b8-6469-48c3-9ba8-96c639269715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:54.279564 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b04b8-6469-48c3-9ba8-96c639269715 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:54.279793 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b04b8-6469-48c3-9ba8-96c639269715 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:54.280315 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b04b8-6469-48c3-9ba8-96c639269715 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:54.291530 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55b04b8-6469-48c3-9ba8-96c639269715 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:54.293550 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55b04b8-6469-48c3-9ba8-96c639269715 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:54.295801 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:54.313766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-d55b04b8-6469-48c3-9ba8-96c639269715 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 13:53:54.313766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:54.313766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:54.313766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:54.313766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:54.313766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:54.313766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:54.313766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:54.313766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:54.313766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:54.313766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:54.313766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:54.313766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:53:54.313766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:53:54.313766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:53:54.313766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:53:54.314879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:54.314879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:54.314879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:53:54.314879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:53:54.314879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:53:54.314879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:53:54.314879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:54.314879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:54.314879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:54.314879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:54.314879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:54.314879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:54.314879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:54.314879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:54.314879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:54.314879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:54.314879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:54.314879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:54.316534 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:54.316534 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:54.316534 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:53:54.316534 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:53:54.316534 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:53:54.316534 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:53:54.316534 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:53:54.316534 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:53:54.316534 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 13:53:54.316534 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:54.318496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 261/515] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:53:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:54.320002 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55b04b8-6469-48c3-9ba8-96c639269715 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:54.349354 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2878f021-0f00-4873-9477-92e357ac3971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:54.350182 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878f021-0f00-4873-9477-92e357ac3971 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:54.350607 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878f021-0f00-4873-9477-92e357ac3971 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:54.351012 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878f021-0f00-4873-9477-92e357ac3971 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:54.475116 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2878f021-0f00-4873-9477-92e357ac3971 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:54.476697 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2878f021-0f00-4873-9477-92e357ac3971 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:54.524893 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55b04b8-6469-48c3-9ba8-96c639269715 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:54.590910 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:54.598667 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dcee1d-d05e-4d13-a1f4-20f01bc3e1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:54.599866 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcee1d-d05e-4d13-a1f4-20f01bc3e1fd None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:54.600488 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcee1d-d05e-4d13-a1f4-20f01bc3e1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:54.601055 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcee1d-d05e-4d13-a1f4-20f01bc3e1fd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:54.615917 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dcee1d-d05e-4d13-a1f4-20f01bc3e1fd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:54.618880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dcee1d-d05e-4d13-a1f4-20f01bc3e1fd None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:54.642311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-41dcee1d-d05e-4d13-a1f4-20f01bc3e1fd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 13:53:54.642311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:54.642311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:54.642311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:54.642311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:54.642311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:54.642311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:54.642311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:54.642311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:54.642311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:54.642311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:54.642311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:54.642311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:53:54.642311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:53:54.642311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:53:54.642311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:53:54.643535 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:54.643535 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:54.643535 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:53:54.643535 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:53:54.643535 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:53:54.643535 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:53:54.643535 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:54.643535 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:54.643535 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:54.643535 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:54.643535 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:54.643535 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:54.643535 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:54.643535 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:54.643535 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:54.643535 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:54.643535 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:54.643535 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:54.650247 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:54.650247 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:54.650247 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:53:54.650247 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:53:54.650247 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:53:54.650247 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:53:54.650247 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:53:54.650247 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:53:54.650247 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 13:53:54.650247 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:54.650247 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 262/516] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 13:53:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:54.650247 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dcee1d-d05e-4d13-a1f4-20f01bc3e1fd None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:54.674601 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 255/517] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:53:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:53:54.674601 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2878f021-0f00-4873-9477-92e357ac3971 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:54.763507 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:54.827070 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dcee1d-d05e-4d13-a1f4-20f01bc3e1fd None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:54.844966 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2878f021-0f00-4873-9477-92e357ac3971 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:54.887472 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb18819e-a049-4d60-ae93-f3d501137f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:54.888613 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18819e-a049-4d60-ae93-f3d501137f02 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:54.889057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18819e-a049-4d60-ae93-f3d501137f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:54.889442 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18819e-a049-4d60-ae93-f3d501137f02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:54.893922 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c689c8e-c10c-4bb5-bf49-b78ffa4f7a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:54.894585 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c689c8e-c10c-4bb5-bf49-b78ffa4f7a06 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:54.894866 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c689c8e-c10c-4bb5-bf49-b78ffa4f7a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:54.895141 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c689c8e-c10c-4bb5-bf49-b78ffa4f7a06 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:54.895987 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c689c8e-c10c-4bb5-bf49-b78ffa4f7a06 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:54.896321 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c689c8e-c10c-4bb5-bf49-b78ffa4f7a06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:54.897825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c689c8e-c10c-4bb5-bf49-b78ffa4f7a06 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:54.911793 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb18819e-a049-4d60-ae93-f3d501137f02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:54.914837 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb18819e-a049-4d60-ae93-f3d501137f02 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:54.918849 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 256/518] 10.222.0.45 () {56 vars in 1137 bytes} [Mon Aug 1 13:53:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:54.919735 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c689c8e-c10c-4bb5-bf49-b78ffa4f7a06 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:54.947690 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-fb18819e-a049-4d60-ae93-f3d501137f02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:53:54.947690 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:54.947690 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:54.947690 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:54.947690 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:54.947690 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:54.947690 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:54.947690 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:54.947690 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:54.947690 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:54.947690 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:54.947690 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:54.947690 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:53:54.947690 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:53:54.947690 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:53:54.948870 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:53:54.948870 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:54.948870 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:54.948870 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:54.948870 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:54.948870 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:54.948870 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:54.948870 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:54.948870 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:54.948870 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:54.948870 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:54.948870 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:54.948870 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:54.948870 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:54.948870 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:54.948870 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:54.948870 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:54.950095 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:53:54.950095 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:53:54.950095 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:53:54.950095 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:54.950095 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 263/519] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:53:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:54.951614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb18819e-a049-4d60-ae93-f3d501137f02 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:54.969163 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:55.080715 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c689c8e-c10c-4bb5-bf49-b78ffa4f7a06 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:55.110700 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb18819e-a049-4d60-ae93-f3d501137f02 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:55.125423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a761fa-43a2-437e-91b1-287ae88a7b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:55.126070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a761fa-43a2-437e-91b1-287ae88a7b35 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:55.126351 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a761fa-43a2-437e-91b1-287ae88a7b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:55.126642 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a761fa-43a2-437e-91b1-287ae88a7b35 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:55.127451 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a761fa-43a2-437e-91b1-287ae88a7b35 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:55.127758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a761fa-43a2-437e-91b1-287ae88a7b35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:55.129290 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a761fa-43a2-437e-91b1-287ae88a7b35 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:55.134697 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:55.152089 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 257/520] 10.222.0.45 () {56 vars in 1139 bytes} [Mon Aug 1 13:53:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:55.153027 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a761fa-43a2-437e-91b1-287ae88a7b35 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:55.182918 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c3c89f-24e2-4b35-b195-2602aa208a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:55.183881 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c3c89f-24e2-4b35-b195-2602aa208a68 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:55.184157 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c3c89f-24e2-4b35-b195-2602aa208a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:55.184463 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c3c89f-24e2-4b35-b195-2602aa208a68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:55.199665 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c3c89f-24e2-4b35-b195-2602aa208a68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:55.202646 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c3c89f-24e2-4b35-b195-2602aa208a68 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:55.221132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-e8c3c89f-24e2-4b35-b195-2602aa208a68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:53:55.221132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:55.221132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:55.221132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:55.221132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:55.221132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:55.221132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:55.221132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:55.221132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:55.221132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:55.221132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:55.221132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:55.221132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:53:55.221132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:53:55.221132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:53:55.222754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:53:55.222754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:55.222754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:55.222754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:55.222754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:55.222754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:55.222754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:55.222754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:55.222754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:55.222754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:55.222754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:55.222754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:55.222754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:55.222754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:55.222754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:55.222754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:55.222754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:55.224461 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:53:55.224461 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:53:55.224461 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:53:55.224461 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:55.224461 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 264/521] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:53:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:55.224461 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c3c89f-24e2-4b35-b195-2602aa208a68 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:55.275256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:55.306252 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a761fa-43a2-437e-91b1-287ae88a7b35 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:55.357860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c3c89f-24e2-4b35-b195-2602aa208a68 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:55.368797 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc20e88f-a33f-4759-abd0-ca4194a520c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:55.369441 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20e88f-a33f-4759-abd0-ca4194a520c8 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:55.369719 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20e88f-a33f-4759-abd0-ca4194a520c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:55.369988 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20e88f-a33f-4759-abd0-ca4194a520c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:55.421453 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88964b43-4e96-4ee4-aa48-326102c38792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:55.422853 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88964b43-4e96-4ee4-aa48-326102c38792 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:55.423523 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88964b43-4e96-4ee4-aa48-326102c38792 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:55.424346 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88964b43-4e96-4ee4-aa48-326102c38792 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:55.425782 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88964b43-4e96-4ee4-aa48-326102c38792 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:55.426540 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88964b43-4e96-4ee4-aa48-326102c38792 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:55.429333 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88964b43-4e96-4ee4-aa48-326102c38792 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:55.449161 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 265/522] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:53:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:55.454236 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88964b43-4e96-4ee4-aa48-326102c38792 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:55.466923 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc20e88f-a33f-4759-abd0-ca4194a520c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:55.470343 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc20e88f-a33f-4759-abd0-ca4194a520c8 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:55.488250 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:55.599687 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:55.606196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88964b43-4e96-4ee4-aa48-326102c38792 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:55.634122 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 258/523] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:53:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:53:55.634122 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc20e88f-a33f-4759-abd0-ca4194a520c8 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:55.665129 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9a373-01d4-404d-b146-5fd787a9350d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:55.665780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9a373-01d4-404d-b146-5fd787a9350d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:55.666053 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9a373-01d4-404d-b146-5fd787a9350d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:55.666321 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9a373-01d4-404d-b146-5fd787a9350d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:55.774407 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9a373-01d4-404d-b146-5fd787a9350d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:55.778160 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9a373-01d4-404d-b146-5fd787a9350d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:55.797239 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc20e88f-a33f-4759-abd0-ca4194a520c8 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:55.818029 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:55.849341 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a7a2c-a649-4434-a7a8-a6c65eac4933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:55.850315 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a7a2c-a649-4434-a7a8-a6c65eac4933 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:55.850601 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a7a2c-a649-4434-a7a8-a6c65eac4933 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:55.850874 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a7a2c-a649-4434-a7a8-a6c65eac4933 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:55.851597 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a7a2c-a649-4434-a7a8-a6c65eac4933 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:55.852477 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a7a2c-a649-4434-a7a8-a6c65eac4933 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:55.854403 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a7a2c-a649-4434-a7a8-a6c65eac4933 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:55.873553 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 259/524] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:53:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:55.874993 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a7a2c-a649-4434-a7a8-a6c65eac4933 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:55.966143 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:55.986487 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 266/525] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:53:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:53:55.986487 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9a373-01d4-404d-b146-5fd787a9350d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:56.061064 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a7a2c-a649-4434-a7a8-a6c65eac4933 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:56.111359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8fbb0f-1f3f-48b5-aba1-2220bc76bdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:56.112015 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8fbb0f-1f3f-48b5-aba1-2220bc76bdb3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:56.112351 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8fbb0f-1f3f-48b5-aba1-2220bc76bdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:56.112588 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8fbb0f-1f3f-48b5-aba1-2220bc76bdb3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:56.120839 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8fbb0f-1f3f-48b5-aba1-2220bc76bdb3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:56.122617 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8fbb0f-1f3f-48b5-aba1-2220bc76bdb3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:56.138118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-ae8fbb0f-1f3f-48b5-aba1-2220bc76bdb3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:53:56.138118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:56.138118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:56.138118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:56.138118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:56.138118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:56.138118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:56.138118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:56.138118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:56.138118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:56.138118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:56.138118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:56.138118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:53:56.138118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:53:56.138118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:53:56.139662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:53:56.139662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:56.139662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:56.139662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:56.139662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:56.139662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:56.139662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:56.139662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:56.139662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:56.139662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:56.139662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:56.139662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:56.139662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:56.139662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:56.139662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:56.139662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:56.139662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:56.141318 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:53:56.141318 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:53:56.141318 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:53:56.141318 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:53:56.141318 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:53:56.141318 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:53:56.141318 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:53:56.141318 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:53:56.147049 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 260/526] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:53:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:56.148683 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8fbb0f-1f3f-48b5-aba1-2220bc76bdb3 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:56.189276 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9a373-01d4-404d-b146-5fd787a9350d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:56.236571 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:56.264185 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96be70e-7e35-4fee-8ed1-d4789f9cc539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:56.265446 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96be70e-7e35-4fee-8ed1-d4789f9cc539 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:56.265953 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96be70e-7e35-4fee-8ed1-d4789f9cc539 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:56.266453 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96be70e-7e35-4fee-8ed1-d4789f9cc539 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:56.289419 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96be70e-7e35-4fee-8ed1-d4789f9cc539 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:56.291828 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96be70e-7e35-4fee-8ed1-d4789f9cc539 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:56.317684 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-e96be70e-7e35-4fee-8ed1-d4789f9cc539 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:53:56.317684 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:56.317684 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:56.317684 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:56.317684 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:56.317684 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:56.317684 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:56.317684 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:56.317684 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:56.317684 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:56.317684 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:56.317684 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:56.317684 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:53:56.317684 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:53:56.317684 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:53:56.319688 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:53:56.319688 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:56.319688 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:56.319688 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:56.319688 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:56.319688 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:56.319688 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:56.319688 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:56.319688 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:56.319688 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:56.319688 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:56.319688 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:56.319688 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:56.319688 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:56.319688 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:56.319688 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:56.319688 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:56.321364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:53:56.321364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:53:56.321364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:53:56.321364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:56.324055 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 267/527] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:53:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:56.325793 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96be70e-7e35-4fee-8ed1-d4789f9cc539 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:56.342826 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8fbb0f-1f3f-48b5-aba1-2220bc76bdb3 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:56.364203 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:56.421863 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572693e-555f-4a39-9d37-8740b866a249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:56.422950 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572693e-555f-4a39-9d37-8740b866a249 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:56.423417 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572693e-555f-4a39-9d37-8740b866a249 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:56.423890 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572693e-555f-4a39-9d37-8740b866a249 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:56.436734 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2572693e-555f-4a39-9d37-8740b866a249 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:56.439576 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2572693e-555f-4a39-9d37-8740b866a249 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:56.454478 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-2572693e-555f-4a39-9d37-8740b866a249 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:53:56.454478 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:56.454478 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:56.454478 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:56.454478 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:56.454478 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:56.454478 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:56.454478 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:56.454478 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:56.454478 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:56.454478 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:56.454478 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:56.454478 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:53:56.454478 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:53:56.454478 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:53:56.456340 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:53:56.456340 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:56.456340 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:56.456340 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:56.456340 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:56.456340 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:56.456340 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:56.456340 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:56.456340 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:56.456340 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:56.456340 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:56.456340 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:56.456340 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:56.456340 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:56.456340 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:56.456340 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:56.456340 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:56.458009 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:53:56.458009 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:53:56.458009 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:53:56.458009 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:53:56.458009 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:53:56.458009 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:53:56.458009 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:53:56.458009 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:53:56.460858 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 261/528] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:53:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:56.461656 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572693e-555f-4a39-9d37-8740b866a249 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:56.530167 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96be70e-7e35-4fee-8ed1-d4789f9cc539 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:56.583655 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649f26d-4c67-4096-a819-bfb9e82e9330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:56.584608 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649f26d-4c67-4096-a819-bfb9e82e9330 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:56.585068 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649f26d-4c67-4096-a819-bfb9e82e9330 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:56.585402 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649f26d-4c67-4096-a819-bfb9e82e9330 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:56.598547 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4649f26d-4c67-4096-a819-bfb9e82e9330 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:56.600088 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4649f26d-4c67-4096-a819-bfb9e82e9330 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:56.618368 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-4649f26d-4c67-4096-a819-bfb9e82e9330 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:53:56.618368 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:56.618368 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:56.618368 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:56.618368 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:56.618368 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:56.618368 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:56.618368 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:56.618368 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:56.618368 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:56.618368 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:56.618368 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:56.618368 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:53:56.618368 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:53:56.618368 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:53:56.620021 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:53:56.620021 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:56.620021 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:56.620021 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:56.620021 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:56.620021 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:56.620021 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:56.620021 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:56.620021 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:56.620021 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:56.620021 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:56.620021 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:56.620021 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:56.620021 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:56.620021 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:56.620021 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:56.620021 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:56.629269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:53:56.629269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:53:56.629269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:53:56.629269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:56.629269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 268/529] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:53:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:56.629269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649f26d-4c67-4096-a819-bfb9e82e9330 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:56.672203 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:56.683160 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572693e-555f-4a39-9d37-8740b866a249 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:56.746261 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ae879-da76-4667-88ae-0d9b34ef7c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:56.746964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ae879-da76-4667-88ae-0d9b34ef7c8c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:56.747245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ae879-da76-4667-88ae-0d9b34ef7c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:56.747528 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ae879-da76-4667-88ae-0d9b34ef7c8c None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:56.748387 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ae879-da76-4667-88ae-0d9b34ef7c8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:56.748701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ae879-da76-4667-88ae-0d9b34ef7c8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:56.750526 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ae879-da76-4667-88ae-0d9b34ef7c8c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:56.774625 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 262/530] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:53:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:56.781556 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ae879-da76-4667-88ae-0d9b34ef7c8c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:56.817609 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:56.831047 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649f26d-4c67-4096-a819-bfb9e82e9330 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:56.901889 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86bc9d-aec5-4c06-8a6b-6f294eb75b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:56.903787 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86bc9d-aec5-4c06-8a6b-6f294eb75b4f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:56.904316 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86bc9d-aec5-4c06-8a6b-6f294eb75b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:56.904810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86bc9d-aec5-4c06-8a6b-6f294eb75b4f None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:56.906048 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff86bc9d-aec5-4c06-8a6b-6f294eb75b4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:56.906356 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff86bc9d-aec5-4c06-8a6b-6f294eb75b4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:56.907817 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff86bc9d-aec5-4c06-8a6b-6f294eb75b4f None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:56.936625 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 269/531] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:56.936625 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86bc9d-aec5-4c06-8a6b-6f294eb75b4f None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:56.959220 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ae879-da76-4667-88ae-0d9b34ef7c8c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:57.025741 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04f88c-9aa7-4dda-99fc-da870c634590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:57.026810 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04f88c-9aa7-4dda-99fc-da870c634590 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:57.027313 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04f88c-9aa7-4dda-99fc-da870c634590 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:57.027737 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04f88c-9aa7-4dda-99fc-da870c634590 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:57.029058 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04f88c-9aa7-4dda-99fc-da870c634590 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:57.029543 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04f88c-9aa7-4dda-99fc-da870c634590 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:57.032139 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04f88c-9aa7-4dda-99fc-da870c634590 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:57.053979 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 263/532] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:53:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:57.062749 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04f88c-9aa7-4dda-99fc-da870c634590 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:57.097934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:57.130000 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86bc9d-aec5-4c06-8a6b-6f294eb75b4f None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:57.206495 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd439e3-3067-4f3a-8658-124223b91d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:57.207530 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd439e3-3067-4f3a-8658-124223b91d7a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:57.208093 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd439e3-3067-4f3a-8658-124223b91d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:57.208666 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd439e3-3067-4f3a-8658-124223b91d7a None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:57.209947 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd439e3-3067-4f3a-8658-124223b91d7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:57.210527 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd439e3-3067-4f3a-8658-124223b91d7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:57.213139 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd439e3-3067-4f3a-8658-124223b91d7a None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:57.235185 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 270/533] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:57.235680 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:57.236682 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd439e3-3067-4f3a-8658-124223b91d7a None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:57.273133 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04f88c-9aa7-4dda-99fc-da870c634590 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:57.336778 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3dde0-28a8-4595-9d80-d50318cbfffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:57.337683 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3dde0-28a8-4595-9d80-d50318cbfffd None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:57.338154 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3dde0-28a8-4595-9d80-d50318cbfffd None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:57.338639 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3dde0-28a8-4595-9d80-d50318cbfffd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:57.340313 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d3dde0-28a8-4595-9d80-d50318cbfffd None admin] RBAC: Adding query filter params (role.id=d17972f7fe97462b8f9f08aed9bc9cb5, scope.project.id=36f063d8f65d494a89dbbd72c53417c8, user.id=5b3adc14dcaf4ac79b43bb97fa1ba1d4) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:57.340910 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d3dde0-28a8-4595-9d80-d50318cbfffd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:57.343986 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d3dde0-28a8-4595-9d80-d50318cbfffd None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:57.356862 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 264/534] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:53:57 2022] GET /identity/v3/role_assignments?user.id=5b3adc14dcaf4ac79b43bb97fa1ba1d4&scope.project.id=36f063d8f65d494a89dbbd72c53417c8&role.id=d17972f7fe97462b8f9f08aed9bc9cb5 => generated 259 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:57.359084 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3dde0-28a8-4595-9d80-d50318cbfffd None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:57.446804 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd439e3-3067-4f3a-8658-124223b91d7a None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:57.510100 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c49e1f-6362-45b1-9602-dfe14fe1decc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:57.511224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c49e1f-6362-45b1-9602-dfe14fe1decc None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:57.511710 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c49e1f-6362-45b1-9602-dfe14fe1decc None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:57.512199 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c49e1f-6362-45b1-9602-dfe14fe1decc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:57.525339 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c49e1f-6362-45b1-9602-dfe14fe1decc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:57.528331 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c49e1f-6362-45b1-9602-dfe14fe1decc None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:57.552122 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3dde0-28a8-4595-9d80-d50318cbfffd None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:57.557572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-24c49e1f-6362-45b1-9602-dfe14fe1decc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:53:57.557572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:57.557572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:57.557572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:57.557572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:57.557572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:57.557572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:57.557572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:57.557572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:57.557572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:57.557572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:57.557572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:57.557572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:53:57.557572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:53:57.557572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:53:57.559488 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:53:57.559488 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:57.559488 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:57.559488 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:57.559488 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:57.559488 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:57.559488 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:57.559488 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:57.559488 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:57.559488 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:57.559488 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:57.559488 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:57.559488 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:57.559488 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:57.559488 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:57.559488 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:57.559488 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:57.561476 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:53:57.561476 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:53:57.561476 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:53:57.561476 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:53:57.561476 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:53:57.561476 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:53:57.561476 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:53:57.561476 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:57.570205 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 271/535] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:53:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:57.570205 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c49e1f-6362-45b1-9602-dfe14fe1decc None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:57.583554 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:57.623293 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae059cf0-d198-4712-988e-8473c600df23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:57.624034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae059cf0-d198-4712-988e-8473c600df23 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:57.626632 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae059cf0-d198-4712-988e-8473c600df23 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:57.626632 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae059cf0-d198-4712-988e-8473c600df23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:57.627367 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae059cf0-d198-4712-988e-8473c600df23 None admin] RBAC: Adding query filter params (role.id=d17972f7fe97462b8f9f08aed9bc9cb5, scope.project.id=36f063d8f65d494a89dbbd72c53417c8, user.id=8f4c3d46ac7243aeb722a4410f40d846) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:57.628025 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae059cf0-d198-4712-988e-8473c600df23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:57.629736 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae059cf0-d198-4712-988e-8473c600df23 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:57.637616 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 265/536] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:53:57 2022] GET /identity/v3/role_assignments?user.id=8f4c3d46ac7243aeb722a4410f40d846&scope.project.id=36f063d8f65d494a89dbbd72c53417c8&role.id=d17972f7fe97462b8f9f08aed9bc9cb5 => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:57.639787 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35d22a-e1e7-476c-9f30-1f7cf67dde49 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:57.640553 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35d22a-e1e7-476c-9f30-1f7cf67dde49 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:57.641069 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35d22a-e1e7-476c-9f30-1f7cf67dde49 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:57.644519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 266/537] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:53:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:53:57.645962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad35d22a-e1e7-476c-9f30-1f7cf67dde49 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:57.680995 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:57.800645 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c49e1f-6362-45b1-9602-dfe14fe1decc None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:57.886994 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00775e2a-cf34-427c-838b-e0acf514fe3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:57.889346 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00775e2a-cf34-427c-838b-e0acf514fe3d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:57.889828 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00775e2a-cf34-427c-838b-e0acf514fe3d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:57.890450 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00775e2a-cf34-427c-838b-e0acf514fe3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:57.890762 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad35d22a-e1e7-476c-9f30-1f7cf67dde49 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:57.909012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00775e2a-cf34-427c-838b-e0acf514fe3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:57.912112 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00775e2a-cf34-427c-838b-e0acf514fe3d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:57.938212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-00775e2a-cf34-427c-838b-e0acf514fe3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:53:57.938212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:57.938212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:57.938212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:57.938212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:57.938212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:57.938212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:57.938212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:57.938212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:57.938212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:57.938212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:57.938212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:57.938212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:53:57.938212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:53:57.938212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:53:57.939807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:53:57.939807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:57.939807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:57.939807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:57.939807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:57.939807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:57.939807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:57.939807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:57.939807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:57.939807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:57.939807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:57.939807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:57.939807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:57.939807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:57.939807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:57.939807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:57.939807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:57.941483 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:53:57.941483 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:53:57.941483 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:53:57.941483 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:53:57.941483 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:53:57.941483 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:53:57.941483 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:53:57.941483 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:57.941483 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 272/538] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:53:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:57.943229 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34848b8e-02fe-4078-97e7-987216683466 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:57.943697 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34848b8e-02fe-4078-97e7-987216683466 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:57.944164 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34848b8e-02fe-4078-97e7-987216683466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:57.980869 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758fa92f-2297-4f15-b455-d83edf954249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:57.983448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fa92f-2297-4f15-b455-d83edf954249 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:57.984423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fa92f-2297-4f15-b455-d83edf954249 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:57.985178 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fa92f-2297-4f15-b455-d83edf954249 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:57.986884 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758fa92f-2297-4f15-b455-d83edf954249 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:57.987693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758fa92f-2297-4f15-b455-d83edf954249 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:57.988195 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-34848b8e-02fe-4078-97e7-987216683466 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:53:57.990418 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-34848b8e-02fe-4078-97e7-987216683466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:53:57.991416 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758fa92f-2297-4f15-b455-d83edf954249 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:58.031734 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 267/539] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:53:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:58.031734 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758fa92f-2297-4f15-b455-d83edf954249 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:58.160313 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:58.180499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 273/540] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:53:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:53:58.181992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34848b8e-02fe-4078-97e7-987216683466 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:58.227170 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:58.284314 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758fa92f-2297-4f15-b455-d83edf954249 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:58.351473 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f73e3-6e37-46f6-84fe-e8c85a5d5077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:58.352595 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f73e3-6e37-46f6-84fe-e8c85a5d5077 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:58.353092 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f73e3-6e37-46f6-84fe-e8c85a5d5077 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:58.353646 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f73e3-6e37-46f6-84fe-e8c85a5d5077 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:58.354975 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221f73e3-6e37-46f6-84fe-e8c85a5d5077 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:58.355529 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221f73e3-6e37-46f6-84fe-e8c85a5d5077 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:58.358424 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221f73e3-6e37-46f6-84fe-e8c85a5d5077 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:58.388799 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 268/541] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:53:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:58.390607 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f73e3-6e37-46f6-84fe-e8c85a5d5077 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:58.393486 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34848b8e-02fe-4078-97e7-987216683466 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:58.451302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225a66f-6ebd-41cd-8efc-72cb1cdd8950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:58.452949 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225a66f-6ebd-41cd-8efc-72cb1cdd8950 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:58.453629 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225a66f-6ebd-41cd-8efc-72cb1cdd8950 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:58.454339 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225a66f-6ebd-41cd-8efc-72cb1cdd8950 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:58.455865 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f225a66f-6ebd-41cd-8efc-72cb1cdd8950 None admin] RBAC: Adding query filter params (role.id=d17972f7fe97462b8f9f08aed9bc9cb5, scope.project.id=36f063d8f65d494a89dbbd72c53417c8, user.id=e7317c8829cb4fa1b30711f1db575cab) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:58.456643 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f225a66f-6ebd-41cd-8efc-72cb1cdd8950 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:58.459269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f225a66f-6ebd-41cd-8efc-72cb1cdd8950 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:58.476515 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 274/542] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:53:58 2022] GET /identity/v3/role_assignments?user.id=e7317c8829cb4fa1b30711f1db575cab&scope.project.id=36f063d8f65d494a89dbbd72c53417c8&role.id=d17972f7fe97462b8f9f08aed9bc9cb5 => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:58.478481 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225a66f-6ebd-41cd-8efc-72cb1cdd8950 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:58.597438 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f73e3-6e37-46f6-84fe-e8c85a5d5077 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:58.655202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:58.677975 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7364b-2790-4050-8c9e-a423992e1aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:58.679133 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7364b-2790-4050-8c9e-a423992e1aab None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:58.679632 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7364b-2790-4050-8c9e-a423992e1aab None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:58.680132 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7364b-2790-4050-8c9e-a423992e1aab None admin] PATH_INFO: `/v3/roles/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:58.683959 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:58.704968 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d7364b-2790-4050-8c9e-a423992e1aab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:53:58.715127 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-c8d7364b-2790-4050-8c9e-a423992e1aab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:53:58.715127 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:58.715127 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:58.715127 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:58.715127 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:58.715127 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:58.715127 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:58.715127 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:58.715127 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:58.715127 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:58.715127 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:58.715127 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:58.715127 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:53:58.715127 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:53:58.715127 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:53:58.715127 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:53:58.716922 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:53:58.716922 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:53:58.716922 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:53:58.716922 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:53:58.716922 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:58.716922 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:58.716922 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:58.716922 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:58.716922 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:58.716922 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:58.716922 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:58.716922 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:58.716922 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:58.716922 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:58.716922 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:58.716922 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:58.718441 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:58.718441 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:58.718441 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:58.718441 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:58.718441 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:53:58.718441 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:53:58.718441 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:53:58.718441 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:53:58.718441 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:53:58.718441 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:53:58.718441 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:53:58.718441 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:53:58.718441 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 269/543] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 13:53:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:58.730298 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7364b-2790-4050-8c9e-a423992e1aab None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:58.828481 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225a66f-6ebd-41cd-8efc-72cb1cdd8950 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:58.891849 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f0da2-3c57-4f51-a891-b314f2273814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:58.894632 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f0da2-3c57-4f51-a891-b314f2273814 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:58.894632 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f0da2-3c57-4f51-a891-b314f2273814 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:58.894632 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f0da2-3c57-4f51-a891-b314f2273814 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:58.895564 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71f0da2-3c57-4f51-a891-b314f2273814 None admin] RBAC: Adding query filter params (role.id=d17972f7fe97462b8f9f08aed9bc9cb5, scope.project.id=36f063d8f65d494a89dbbd72c53417c8, user.id=6dd44b7a78594288b6aabb393fd6324a) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:58.896034 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71f0da2-3c57-4f51-a891-b314f2273814 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:58.898449 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71f0da2-3c57-4f51-a891-b314f2273814 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:58.909054 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 275/544] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:53:58 2022] GET /identity/v3/role_assignments?user.id=6dd44b7a78594288b6aabb393fd6324a&scope.project.id=36f063d8f65d494a89dbbd72c53417c8&role.id=d17972f7fe97462b8f9f08aed9bc9cb5 => generated 259 bytes in 431 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:58.966122 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7364b-2790-4050-8c9e-a423992e1aab None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:59.066391 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08836e30-da52-4407-a1dd-c56257f975c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:59.067576 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08836e30-da52-4407-a1dd-c56257f975c0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:59.067979 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08836e30-da52-4407-a1dd-c56257f975c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:59.068506 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08836e30-da52-4407-a1dd-c56257f975c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:59.070095 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08836e30-da52-4407-a1dd-c56257f975c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:59.071526 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08836e30-da52-4407-a1dd-c56257f975c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:59.077312 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08836e30-da52-4407-a1dd-c56257f975c0 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:59.103901 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 270/545] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 13:53:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:59.115332 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f0da2-3c57-4f51-a891-b314f2273814 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:59.331729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f0da2-3c57-4f51-a891-b314f2273814 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:59.380390 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:59.406240 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e587fe89-2755-419a-890d-37a8cd484d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:59.407398 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587fe89-2755-419a-890d-37a8cd484d23 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:59.407908 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587fe89-2755-419a-890d-37a8cd484d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:59.408405 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587fe89-2755-419a-890d-37a8cd484d23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:59.432023 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e587fe89-2755-419a-890d-37a8cd484d23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:59.434812 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e587fe89-2755-419a-890d-37a8cd484d23 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:59.474502 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-e587fe89-2755-419a-890d-37a8cd484d23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:53:59.474502 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:53:59.474502 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:53:59.474502 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:53:59.474502 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:53:59.474502 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:53:59.474502 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:53:59.474502 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:53:59.474502 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:53:59.474502 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:53:59.474502 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:53:59.474502 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:53:59.474502 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:53:59.474502 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:53:59.474502 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:53:59.477556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:53:59.477556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:53:59.477556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:53:59.477556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:53:59.477556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:53:59.477556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:53:59.477556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:53:59.477556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:53:59.477556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:53:59.477556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:53:59.477556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:53:59.477556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:53:59.477556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:53:59.477556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:53:59.477556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:53:59.477556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:53:59.477556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:53:59.480786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:53:59.480786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:53:59.480786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:53:59.480786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:53:59.481655 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 276/546] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:53:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:53:59.490826 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08836e30-da52-4407-a1dd-c56257f975c0 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:53:59.655687 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:53:59.744369 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08836e30-da52-4407-a1dd-c56257f975c0 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:53:59.818058 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e4e436-b993-4722-ad40-aab8fe7bd77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:53:59.819192 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4e436-b993-4722-ad40-aab8fe7bd77d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:53:59.819700 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4e436-b993-4722-ad40-aab8fe7bd77d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:53:59.820202 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4e436-b993-4722-ad40-aab8fe7bd77d None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:53:59.821581 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e4e436-b993-4722-ad40-aab8fe7bd77d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:53:59.822145 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e4e436-b993-4722-ad40-aab8fe7bd77d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:53:59.824854 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e4e436-b993-4722-ad40-aab8fe7bd77d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:53:59.850556 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 271/547] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:53:59.862254 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e587fe89-2755-419a-890d-37a8cd484d23 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:00.103466 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e587fe89-2755-419a-890d-37a8cd484d23 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:00.177593 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c52b8-1056-450d-9738-625d196577f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:00.178717 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c52b8-1056-450d-9738-625d196577f0 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:00.179256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c52b8-1056-450d-9738-625d196577f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:00.179700 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c52b8-1056-450d-9738-625d196577f0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:00.204055 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00c52b8-1056-450d-9738-625d196577f0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:00.207022 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00c52b8-1056-450d-9738-625d196577f0 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:00.239844 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-c00c52b8-1056-450d-9738-625d196577f0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 13:54:00.239844 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:00.239844 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:00.239844 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:00.239844 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:00.239844 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:00.239844 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:00.239844 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:00.239844 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:00.239844 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:00.239844 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:00.239844 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:00.239844 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:54:00.239844 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:54:00.239844 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:54:00.239844 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:54:00.241709 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:00.241709 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:00.241709 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:54:00.241709 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:00.241709 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:54:00.241709 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:00.241709 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:00.241709 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:00.241709 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:00.241709 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:00.241709 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:00.241709 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:00.241709 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:00.241709 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:00.241709 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:00.241709 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:00.241709 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:00.241709 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:00.243302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:00.243302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:00.243302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:54:00.243302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:54:00.243302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:54:00.243302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:54:00.243302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:54:00.243302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:54:00.243302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 13:54:00.243302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:00.243302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 277/548] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:53:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:00.253638 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e4e436-b993-4722-ad40-aab8fe7bd77d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:00.464653 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e4e436-b993-4722-ad40-aab8fe7bd77d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:00.496742 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:00.530926 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcabff-872a-4565-a561-bdca5e4f792e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:00.531960 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcabff-872a-4565-a561-bdca5e4f792e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:00.532482 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcabff-872a-4565-a561-bdca5e4f792e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:00.532974 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcabff-872a-4565-a561-bdca5e4f792e None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:00.534546 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fcabff-872a-4565-a561-bdca5e4f792e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:00.535086 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fcabff-872a-4565-a561-bdca5e4f792e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:00.537999 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fcabff-872a-4565-a561-bdca5e4f792e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:00.576065 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 272/549] 10.222.0.45 () {56 vars in 1135 bytes} [Mon Aug 1 13:54:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:00.585131 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c52b8-1056-450d-9738-625d196577f0 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:00.679989 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:00.850811 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c52b8-1056-450d-9738-625d196577f0 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:00.938919 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6bf42b-fb3a-4b64-969f-0e012d66979c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:00.940134 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6bf42b-fb3a-4b64-969f-0e012d66979c None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:00.940492 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6bf42b-fb3a-4b64-969f-0e012d66979c None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:00.940972 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6bf42b-fb3a-4b64-969f-0e012d66979c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:01.002980 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3117430-7354-4949-8516-7800409c772b None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:01.003399 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3117430-7354-4949-8516-7800409c772b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:01.003760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3117430-7354-4949-8516-7800409c772b None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:01.005563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 273/550] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:54:01.012156 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762852ec-e3c3-43d0-a6cf-d86508486619 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:01.013024 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762852ec-e3c3-43d0-a6cf-d86508486619 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:01.013775 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762852ec-e3c3-43d0-a6cf-d86508486619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:01.075290 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-762852ec-e3c3-43d0-a6cf-d86508486619 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:01.077146 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-762852ec-e3c3-43d0-a6cf-d86508486619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:01.125695 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6bf42b-fb3a-4b64-969f-0e012d66979c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:01.129350 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6bf42b-fb3a-4b64-969f-0e012d66979c None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:01.215903 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:01.253765 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 274/551] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:54:01.321561 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762852ec-e3c3-43d0-a6cf-d86508486619 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:01.346792 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 278/552] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:54:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 763 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:54:01.365509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6bf42b-fb3a-4b64-969f-0e012d66979c None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:01.542442 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6bf42b-fb3a-4b64-969f-0e012d66979c None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:01.619100 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762852ec-e3c3-43d0-a6cf-d86508486619 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:01.632881 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:01.654230 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05eb453-862f-4543-82c3-4d52459d42f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:01.655294 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05eb453-862f-4543-82c3-4d52459d42f0 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:01.655662 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05eb453-862f-4543-82c3-4d52459d42f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:01.655951 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05eb453-862f-4543-82c3-4d52459d42f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:01.659982 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:01.670845 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05eb453-862f-4543-82c3-4d52459d42f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:01.673316 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05eb453-862f-4543-82c3-4d52459d42f0 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:01.690467 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-b05eb453-862f-4543-82c3-4d52459d42f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:01.690467 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:01.690467 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:01.690467 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:01.690467 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:01.690467 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:01.690467 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:01.690467 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:01.690467 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:01.690467 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:01.690467 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:01.690467 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:01.690467 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:01.690467 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:01.690467 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:01.692217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:01.692217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:01.692217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:01.692217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:01.692217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:01.692217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:01.692217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:01.692217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:01.692217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:01.692217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:01.692217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:01.692217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:01.692217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:01.692217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:01.692217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:01.692217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:01.692217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:01.693911 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:01.693911 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:01.693911 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:01.693911 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:01.693911 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 279/553] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:54:01.713408 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05eb453-862f-4543-82c3-4d52459d42f0 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:01.726120 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690eb72-ddf9-460a-8c15-460428e338c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:01.727067 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690eb72-ddf9-460a-8c15-460428e338c6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:01.727487 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690eb72-ddf9-460a-8c15-460428e338c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:01.727856 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690eb72-ddf9-460a-8c15-460428e338c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:01.746013 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4690eb72-ddf9-460a-8c15-460428e338c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:54:01.754091 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-4690eb72-ddf9-460a-8c15-460428e338c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:54:01.754091 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:01.754091 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:01.754091 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:01.754091 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:01.754091 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:01.754091 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:01.754091 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:01.754091 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:01.754091 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:01.754091 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:01.754091 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:01.754091 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:54:01.754091 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:54:01.754091 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:54:01.754091 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:54:01.757146 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:54:01.757146 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:54:01.757146 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:54:01.757146 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:54:01.757146 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:01.757146 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:01.757146 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:01.757146 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:01.757146 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:01.757146 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:01.757146 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:01.757146 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:01.757146 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:01.757146 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:01.757146 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:01.757146 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:01.764825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:01.764825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:01.764825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:01.764825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:01.764825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:54:01.764825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:54:01.764825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:54:01.764825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:54:01.764825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:54:01.764825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:54:01.764825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:54:01.764825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:01.764825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 275/554] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 13:54:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 435 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:01.764825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b134a3e-13e7-4622-b25d-984d9f33dc2f None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:01.767959 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b134a3e-13e7-4622-b25d-984d9f33dc2f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:01.767959 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b134a3e-13e7-4622-b25d-984d9f33dc2f None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:01.767959 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 276/555] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:01.772955 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b134a3e-13e7-4622-b25d-984d9f33dc2f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:01.876080 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05eb453-862f-4543-82c3-4d52459d42f0 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:01.931845 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18e5841-2ec5-4783-a065-545ed6371542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:01.932843 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e5841-2ec5-4783-a065-545ed6371542 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:01.933243 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e5841-2ec5-4783-a065-545ed6371542 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:01.933628 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e5841-2ec5-4783-a065-545ed6371542 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:01.934712 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18e5841-2ec5-4783-a065-545ed6371542 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:01.935147 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18e5841-2ec5-4783-a065-545ed6371542 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:01.937446 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18e5841-2ec5-4783-a065-545ed6371542 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:01.950691 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b134a3e-13e7-4622-b25d-984d9f33dc2f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:01.958573 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 280/556] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:54:01.960536 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aca7f0-f947-4540-9cc9-10155bdc8ac9 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:01.960945 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aca7f0-f947-4540-9cc9-10155bdc8ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:01.961353 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aca7f0-f947-4540-9cc9-10155bdc8ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:01.992615 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-30aca7f0-f947-4540-9cc9-10155bdc8ac9 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:01.994504 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-30aca7f0-f947-4540-9cc9-10155bdc8ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:02.025723 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c8de2-2d5f-4d53-a36f-6e04aec17bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:02.026480 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c8de2-2d5f-4d53-a36f-6e04aec17bda None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:02.026760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c8de2-2d5f-4d53-a36f-6e04aec17bda None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:02.027020 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c8de2-2d5f-4d53-a36f-6e04aec17bda None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:02.027846 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c8de2-2d5f-4d53-a36f-6e04aec17bda None admin] RBAC: Adding query filter params (name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:02.028222 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c8de2-2d5f-4d53-a36f-6e04aec17bda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:02.029750 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c8de2-2d5f-4d53-a36f-6e04aec17bda None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:02.047063 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 277/557] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 13:54:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:02.048785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c8de2-2d5f-4d53-a36f-6e04aec17bda None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:02.080204 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:02.085747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:02.128475 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 281/558] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:54:02.130010 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aca7f0-f947-4540-9cc9-10155bdc8ac9 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:02.197435 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c8de2-2d5f-4d53-a36f-6e04aec17bda None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:02.255826 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6bf53-149e-4e45-b591-65588d94cf52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:02.256858 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6bf53-149e-4e45-b591-65588d94cf52 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:02.257096 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6bf53-149e-4e45-b591-65588d94cf52 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:02.257329 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6bf53-149e-4e45-b591-65588d94cf52 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:02.265685 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac6bf53-149e-4e45-b591-65588d94cf52 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:02.267187 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac6bf53-149e-4e45-b591-65588d94cf52 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:02.282978 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-cac6bf53-149e-4e45-b591-65588d94cf52 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:02.282978 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:02.282978 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:02.282978 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:02.282978 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:02.282978 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:02.282978 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:02.282978 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:02.282978 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:02.282978 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:02.282978 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:02.282978 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:02.282978 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:54:02.282978 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:54:02.282978 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:54:02.285563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:54:02.285563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:02.285563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:02.285563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:02.285563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:02.285563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:02.285563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:02.285563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:02.285563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:02.285563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:02.285563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:02.285563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:02.285563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:02.285563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:02.285563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:02.285563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:02.285563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:02.287747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:54:02.287747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:54:02.287747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:54:02.287747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:54:02.287747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:54:02.287747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:54:02.287747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:02.287747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:02.292495 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 278/559] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:54:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:02.294330 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6bf53-149e-4e45-b591-65588d94cf52 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:02.297758 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aca7f0-f947-4540-9cc9-10155bdc8ac9 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:02.358584 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940bd7bb-7d4b-4d2a-97d7-a127b6b8ea16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:02.359608 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bd7bb-7d4b-4d2a-97d7-a127b6b8ea16 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:02.359911 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bd7bb-7d4b-4d2a-97d7-a127b6b8ea16 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:02.360228 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bd7bb-7d4b-4d2a-97d7-a127b6b8ea16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:02.375862 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940bd7bb-7d4b-4d2a-97d7-a127b6b8ea16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:02.378049 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940bd7bb-7d4b-4d2a-97d7-a127b6b8ea16 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:02.400295 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-940bd7bb-7d4b-4d2a-97d7-a127b6b8ea16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:02.400295 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:02.400295 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:02.400295 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:02.400295 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:02.400295 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:02.400295 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:02.400295 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:02.400295 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:02.400295 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:02.400295 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:02.400295 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:02.400295 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:02.400295 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:02.400295 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:02.402414 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:02.402414 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:02.402414 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:02.402414 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:02.402414 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:02.402414 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:02.402414 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:02.402414 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:02.402414 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:02.402414 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:02.402414 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:02.402414 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:02.402414 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:02.402414 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:02.402414 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:02.402414 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:02.402414 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:02.404916 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:02.404916 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:02.404916 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:02.404916 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:02.408332 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 282/560] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:02.408332 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940bd7bb-7d4b-4d2a-97d7-a127b6b8ea16 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:02.453286 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:02.467238 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:02.495972 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6bf53-149e-4e45-b591-65588d94cf52 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:02.546504 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eadaa76-799c-4be8-887d-8105a0993eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:02.547646 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eadaa76-799c-4be8-887d-8105a0993eda None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:02.548062 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eadaa76-799c-4be8-887d-8105a0993eda None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:02.548861 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eadaa76-799c-4be8-887d-8105a0993eda None admin] PATH_INFO: `/v3/roles/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:02.567615 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eadaa76-799c-4be8-887d-8105a0993eda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:54:02.576751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-7eadaa76-799c-4be8-887d-8105a0993eda None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:54:02.576751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:02.576751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:02.576751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:02.576751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:02.576751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:02.576751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:02.576751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:02.576751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:02.576751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:02.576751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:02.576751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:02.576751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:54:02.576751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:54:02.576751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:54:02.576751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:54:02.578338 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:54:02.578338 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:54:02.578338 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:54:02.578338 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:54:02.578338 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:02.578338 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:02.578338 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:02.578338 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:02.578338 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:02.578338 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:02.578338 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:02.578338 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:02.578338 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:02.578338 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:02.578338 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:02.578338 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:02.585210 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:02.585210 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:02.585210 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:02.585210 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:02.585210 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:54:02.585210 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:54:02.585210 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:54:02.585210 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:54:02.585210 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:54:02.585210 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:54:02.585210 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:54:02.585210 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:02.585210 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 279/561] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 13:54:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:02.586477 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940bd7bb-7d4b-4d2a-97d7-a127b6b8ea16 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:02.586627 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eadaa76-799c-4be8-887d-8105a0993eda None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:02.640516 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e2d8b-7da6-4fcb-955c-af330d1d1ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:02.641761 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e2d8b-7da6-4fcb-955c-af330d1d1ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:02.642258 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e2d8b-7da6-4fcb-955c-af330d1d1ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:02.642723 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e2d8b-7da6-4fcb-955c-af330d1d1ac7 None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:02.644429 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e2d8b-7da6-4fcb-955c-af330d1d1ac7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:02.644959 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e2d8b-7da6-4fcb-955c-af330d1d1ac7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:02.647481 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e2d8b-7da6-4fcb-955c-af330d1d1ac7 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:02.680848 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 283/562] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:54:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:02.682757 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40327e-a1b6-47ed-aa62-46bc128cbe79 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:02.683042 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40327e-a1b6-47ed-aa62-46bc128cbe79 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:02.683392 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40327e-a1b6-47ed-aa62-46bc128cbe79 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:02.685956 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 284/563] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:02.688187 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40327e-a1b6-47ed-aa62-46bc128cbe79 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:02.779801 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eadaa76-799c-4be8-887d-8105a0993eda None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:02.832929 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc420cfa-05a8-435b-a5ae-eba99f2838c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:02.833706 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc420cfa-05a8-435b-a5ae-eba99f2838c1 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:02.834079 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc420cfa-05a8-435b-a5ae-eba99f2838c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:02.834392 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc420cfa-05a8-435b-a5ae-eba99f2838c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:02.835330 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc420cfa-05a8-435b-a5ae-eba99f2838c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:02.835643 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc420cfa-05a8-435b-a5ae-eba99f2838c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:02.837103 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc420cfa-05a8-435b-a5ae-eba99f2838c1 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:02.844877 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:02.855626 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 280/564] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:02.856773 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc420cfa-05a8-435b-a5ae-eba99f2838c1 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:02.859442 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40327e-a1b6-47ed-aa62-46bc128cbe79 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:02.867603 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:02.919653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d662b9a0-85da-452e-a513-c69ca9fed90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:02.920980 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662b9a0-85da-452e-a513-c69ca9fed90a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:02.921302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662b9a0-85da-452e-a513-c69ca9fed90a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:02.921604 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662b9a0-85da-452e-a513-c69ca9fed90a None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:02.922380 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d662b9a0-85da-452e-a513-c69ca9fed90a None admin] RBAC: Adding query filter params (name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:02.922688 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d662b9a0-85da-452e-a513-c69ca9fed90a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:02.924193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d662b9a0-85da-452e-a513-c69ca9fed90a None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:02.934516 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 285/565] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 13:54:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:02.935862 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33642402-0a52-479e-89e2-792f6b945d25 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:02.936078 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33642402-0a52-479e-89e2-792f6b945d25 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:02.936401 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33642402-0a52-479e-89e2-792f6b945d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:02.959931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-33642402-0a52-479e-89e2-792f6b945d25 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:02.962056 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-33642402-0a52-479e-89e2-792f6b945d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:03.007467 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc420cfa-05a8-435b-a5ae-eba99f2838c1 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:03.050330 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f363eae2-1cc9-4e7c-83ab-42c8287aec5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:03.050960 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363eae2-1cc9-4e7c-83ab-42c8287aec5d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:03.051238 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363eae2-1cc9-4e7c-83ab-42c8287aec5d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:03.051502 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363eae2-1cc9-4e7c-83ab-42c8287aec5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:03.052387 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f363eae2-1cc9-4e7c-83ab-42c8287aec5d None admin] RBAC: Adding query filter params (role.id=d17972f7fe97462b8f9f08aed9bc9cb5, scope.project.id=36f063d8f65d494a89dbbd72c53417c8, user.id=0cda7f365ba0405daaf86870960f12d2) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:03.052702 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f363eae2-1cc9-4e7c-83ab-42c8287aec5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:03.054233 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f363eae2-1cc9-4e7c-83ab-42c8287aec5d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:03.060696 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 281/566] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:54:02 2022] GET /identity/v3/role_assignments?user.id=0cda7f365ba0405daaf86870960f12d2&scope.project.id=36f063d8f65d494a89dbbd72c53417c8&role.id=d17972f7fe97462b8f9f08aed9bc9cb5 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:03.062115 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e207dc-2e8d-4018-b9b6-66e1d7baf39e None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:03.062360 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e207dc-2e8d-4018-b9b6-66e1d7baf39e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:03.062631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e207dc-2e8d-4018-b9b6-66e1d7baf39e None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:03.064037 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 282/567] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:03.064928 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e207dc-2e8d-4018-b9b6-66e1d7baf39e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:03.075770 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 286/568] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:54:03.078207 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33642402-0a52-479e-89e2-792f6b945d25 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:03.194818 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:03.220067 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:03.248243 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33642402-0a52-479e-89e2-792f6b945d25 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:03.251794 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e207dc-2e8d-4018-b9b6-66e1d7baf39e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:03.296478 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02de071-4e06-4676-8c3f-ff914e0a3b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:03.297149 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02de071-4e06-4676-8c3f-ff914e0a3b88 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:03.297428 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02de071-4e06-4676-8c3f-ff914e0a3b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:03.297703 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02de071-4e06-4676-8c3f-ff914e0a3b88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:03.311310 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02de071-4e06-4676-8c3f-ff914e0a3b88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:03.312912 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02de071-4e06-4676-8c3f-ff914e0a3b88 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:03.328616 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fcf473-f74f-4d9c-b6a3-25abdf1f99c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:03.329630 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fcf473-f74f-4d9c-b6a3-25abdf1f99c1 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:03.330085 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fcf473-f74f-4d9c-b6a3-25abdf1f99c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:03.330532 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fcf473-f74f-4d9c-b6a3-25abdf1f99c1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:03.335833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-e02de071-4e06-4676-8c3f-ff914e0a3b88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:03.335833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:03.335833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:03.335833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:03.335833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:03.335833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:03.335833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:03.335833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:03.335833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:03.335833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:03.335833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:03.335833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:03.335833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:03.335833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:03.335833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:03.337612 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:03.337612 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:03.337612 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:03.337612 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:03.337612 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:03.337612 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:03.337612 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:03.337612 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:03.337612 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:03.337612 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:03.337612 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:03.337612 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:03.337612 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:03.337612 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:03.337612 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:03.337612 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:03.337612 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:03.339560 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:03.339560 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:03.339560 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:03.339560 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:03.344007 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 287/569] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:03.345938 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fcf473-f74f-4d9c-b6a3-25abdf1f99c1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:03.346282 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3945d46-bde5-4a80-93d0-c569b489ff34 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:03.346746 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3945d46-bde5-4a80-93d0-c569b489ff34 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:03.347158 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3945d46-bde5-4a80-93d0-c569b489ff34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:03.349825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fcf473-f74f-4d9c-b6a3-25abdf1f99c1 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:03.367661 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-91fcf473-f74f-4d9c-b6a3-25abdf1f99c1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:54:03.367661 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:03.367661 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:03.367661 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:03.367661 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:03.367661 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:03.367661 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:03.367661 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:03.367661 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:03.367661 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:03.367661 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:03.367661 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:03.367661 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:54:03.367661 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:54:03.367661 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:54:03.367661 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:54:03.369232 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:03.369232 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:03.369232 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:54:03.369232 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:03.369232 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:54:03.369232 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:03.369232 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:03.369232 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:03.369232 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:03.369232 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:03.369232 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:03.369232 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:03.369232 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:03.369232 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:03.369232 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:03.369232 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:03.369232 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:03.369232 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:03.371510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:03.371510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:03.371510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:54:03.371510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:54:03.371510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:54:03.371510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:54:03.371510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:54:03.371510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:54:03.371510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:54:03.371510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:03.374185 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 283/570] 10.222.0.45 () {56 vars in 1093 bytes} [Mon Aug 1 13:54:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:03.375623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fcf473-f74f-4d9c-b6a3-25abdf1f99c1 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:03.382771 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-c3945d46-bde5-4a80-93d0-c569b489ff34 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:03.390649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c3945d46-bde5-4a80-93d0-c569b489ff34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:03.518319 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 288/571] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:54:03.519933 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3945d46-bde5-4a80-93d0-c569b489ff34 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:03.567408 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fcf473-f74f-4d9c-b6a3-25abdf1f99c1 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:03.621887 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:03.622500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:03.632196 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e7506-363d-4560-87d1-4be1cdefe14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:03.633595 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e7506-363d-4560-87d1-4be1cdefe14f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:03.635287 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e7506-363d-4560-87d1-4be1cdefe14f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:03.635654 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e7506-363d-4560-87d1-4be1cdefe14f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:03.654257 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e7506-363d-4560-87d1-4be1cdefe14f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:54:03.662075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-909e7506-363d-4560-87d1-4be1cdefe14f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:54:03.662075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:03.662075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:03.662075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:03.662075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:03.662075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:03.662075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:03.662075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:03.662075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:03.662075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:03.662075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:03.662075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:03.662075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:54:03.662075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:54:03.662075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:54:03.662075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:54:03.663291 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:54:03.663291 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:54:03.663291 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:54:03.663291 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:54:03.663291 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:03.663291 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:03.663291 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:03.663291 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:03.663291 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:03.663291 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:03.663291 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:03.663291 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:03.663291 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:03.663291 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:03.663291 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:03.663291 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:03.665230 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:03.665230 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:03.665230 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:03.665230 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:03.665230 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:54:03.665230 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:54:03.665230 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:54:03.665230 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:54:03.665230 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:54:03.665230 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:54:03.665230 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:54:03.665230 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:03.665230 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 284/572] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 13:54:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:03.666638 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e7506-363d-4560-87d1-4be1cdefe14f None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:03.686263 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3945d46-bde5-4a80-93d0-c569b489ff34 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:03.741513 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446db5ea-1d2f-4274-9400-e04f29280110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:03.742306 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446db5ea-1d2f-4274-9400-e04f29280110 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:03.742664 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446db5ea-1d2f-4274-9400-e04f29280110 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:03.742963 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446db5ea-1d2f-4274-9400-e04f29280110 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:03.743818 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446db5ea-1d2f-4274-9400-e04f29280110 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:03.744127 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446db5ea-1d2f-4274-9400-e04f29280110 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:03.745706 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446db5ea-1d2f-4274-9400-e04f29280110 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:03.766624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 289/573] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:03.766624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446db5ea-1d2f-4274-9400-e04f29280110 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:03.852605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e7506-363d-4560-87d1-4be1cdefe14f None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:03.925584 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b4e86-3bc0-4b90-87a7-55bbf51ca098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:03.926648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b4e86-3bc0-4b90-87a7-55bbf51ca098 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:03.927138 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b4e86-3bc0-4b90-87a7-55bbf51ca098 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:03.927627 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b4e86-3bc0-4b90-87a7-55bbf51ca098 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:03.929155 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b4e86-3bc0-4b90-87a7-55bbf51ca098 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:03.929499 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b4e86-3bc0-4b90-87a7-55bbf51ca098 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:03.932519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b4e86-3bc0-4b90-87a7-55bbf51ca098 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:03.971020 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 285/574] 10.222.0.45 () {56 vars in 1133 bytes} [Mon Aug 1 13:54:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:03.971020 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b4e86-3bc0-4b90-87a7-55bbf51ca098 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:04.003756 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446db5ea-1d2f-4274-9400-e04f29280110 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:04.055023 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:04.085853 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:04.091600 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c573c7-ee49-45f4-97a5-48730a4abb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:04.092864 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c573c7-ee49-45f4-97a5-48730a4abb9f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:04.093463 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c573c7-ee49-45f4-97a5-48730a4abb9f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:04.093937 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c573c7-ee49-45f4-97a5-48730a4abb9f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:04.117394 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c573c7-ee49-45f4-97a5-48730a4abb9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:54:04.128109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-d7c573c7-ee49-45f4-97a5-48730a4abb9f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:54:04.128109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:04.128109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:04.128109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:04.128109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:04.128109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:04.128109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:04.128109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:04.128109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:04.128109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:04.128109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:04.128109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:04.128109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:54:04.128109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:54:04.128109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:54:04.128109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:54:04.129205 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:54:04.129205 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:54:04.129205 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:54:04.129205 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:54:04.129205 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:04.129205 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:04.129205 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:04.129205 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:04.129205 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:04.129205 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:04.129205 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:04.129205 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:04.129205 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:04.129205 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:04.129205 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:04.129205 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:04.130053 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:04.130053 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:04.130053 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:04.130053 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:04.130053 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:54:04.130053 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:54:04.130053 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:54:04.130053 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:54:04.130053 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:54:04.130053 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:54:04.130053 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:54:04.130053 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:04.130733 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 290/575] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 13:54:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:04.131910 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c573c7-ee49-45f4-97a5-48730a4abb9f None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:04.198634 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b4e86-3bc0-4b90-87a7-55bbf51ca098 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:04.249791 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729c1ee-f43d-4c37-8a5a-b07e620a0158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:04.250815 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729c1ee-f43d-4c37-8a5a-b07e620a0158 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:04.251281 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729c1ee-f43d-4c37-8a5a-b07e620a0158 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:04.251744 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729c1ee-f43d-4c37-8a5a-b07e620a0158 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:04.253047 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f729c1ee-f43d-4c37-8a5a-b07e620a0158 None admin] RBAC: Adding query filter params (name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:04.253556 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f729c1ee-f43d-4c37-8a5a-b07e620a0158 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:04.255752 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f729c1ee-f43d-4c37-8a5a-b07e620a0158 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:04.270969 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 286/576] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 13:54:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:04.271955 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729c1ee-f43d-4c37-8a5a-b07e620a0158 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:04.334569 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c573c7-ee49-45f4-97a5-48730a4abb9f None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:04.402716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26950576-9d73-4061-a523-2c5c3b9eeeaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:04.404354 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26950576-9d73-4061-a523-2c5c3b9eeeaf None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:04.405168 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26950576-9d73-4061-a523-2c5c3b9eeeaf None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:04.405996 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26950576-9d73-4061-a523-2c5c3b9eeeaf None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:04.419505 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26950576-9d73-4061-a523-2c5c3b9eeeaf None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:04.422603 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26950576-9d73-4061-a523-2c5c3b9eeeaf None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:04.442658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-26950576-9d73-4061-a523-2c5c3b9eeeaf None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 13:54:04.442658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:04.442658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:04.442658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:04.442658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:04.442658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:04.442658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:04.442658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:04.442658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:04.442658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:04.442658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:04.442658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:04.442658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:54:04.442658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:54:04.442658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:54:04.442658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:54:04.444653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:04.444653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:04.444653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:54:04.444653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:04.444653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:54:04.444653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:04.444653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:04.444653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:04.444653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:04.444653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:04.444653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:04.444653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:04.444653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:04.444653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:04.444653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:04.444653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:04.444653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:04.444653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:04.446754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:04.446754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:04.446754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:54:04.446754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:54:04.446754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:54:04.446754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:54:04.446754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:54:04.446754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:54:04.446754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 13:54:04.446754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:04.450368 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 291/577] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:54:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:04.452309 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26950576-9d73-4061-a523-2c5c3b9eeeaf None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:04.482912 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729c1ee-f43d-4c37-8a5a-b07e620a0158 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:04.505244 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:04.566807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:04.574042 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d7ca46-d8dd-4b6a-b8c3-b13b8327cedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:04.575310 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7ca46-d8dd-4b6a-b8c3-b13b8327cedb None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:04.575689 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7ca46-d8dd-4b6a-b8c3-b13b8327cedb None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:04.576160 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7ca46-d8dd-4b6a-b8c3-b13b8327cedb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:04.593356 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d7ca46-d8dd-4b6a-b8c3-b13b8327cedb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:04.595083 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d7ca46-d8dd-4b6a-b8c3-b13b8327cedb None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:04.614913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-c9d7ca46-d8dd-4b6a-b8c3-b13b8327cedb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:04.614913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:04.614913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:04.614913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:04.614913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:04.614913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:04.614913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:04.614913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:04.614913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:04.614913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:04.614913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:04.614913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:04.614913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:04.614913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:04.614913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:04.624368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:04.624368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:04.624368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:04.624368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:04.624368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:04.624368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:04.624368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:04.624368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:04.624368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:04.624368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:04.624368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:04.624368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:04.624368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:04.624368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:04.624368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:04.624368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:04.624368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:04.626406 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:04.626406 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:04.626406 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:04.626406 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:04.626406 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 287/578] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:04.626406 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d7ca46-d8dd-4b6a-b8c3-b13b8327cedb None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:04.700531 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26950576-9d73-4061-a523-2c5c3b9eeeaf None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:04.776598 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d398310-9d8d-40d0-bd99-d42a7a20710e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:04.777755 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d398310-9d8d-40d0-bd99-d42a7a20710e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:04.778483 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d398310-9d8d-40d0-bd99-d42a7a20710e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:04.778789 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d398310-9d8d-40d0-bd99-d42a7a20710e None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:04.780218 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d398310-9d8d-40d0-bd99-d42a7a20710e None admin] RBAC: Adding query filter params (name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:04.780898 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d398310-9d8d-40d0-bd99-d42a7a20710e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:04.783467 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d398310-9d8d-40d0-bd99-d42a7a20710e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:04.804760 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 292/579] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 13:54:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:04.804760 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d398310-9d8d-40d0-bd99-d42a7a20710e None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:04.809963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d7ca46-d8dd-4b6a-b8c3-b13b8327cedb None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:04.875349 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a0225-8483-4d5a-b403-71bde1d20f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:04.876558 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a0225-8483-4d5a-b403-71bde1d20f64 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:04.877064 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a0225-8483-4d5a-b403-71bde1d20f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:04.877560 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a0225-8483-4d5a-b403-71bde1d20f64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:04.900506 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879a0225-8483-4d5a-b403-71bde1d20f64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:04.903094 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879a0225-8483-4d5a-b403-71bde1d20f64 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:04.931575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-879a0225-8483-4d5a-b403-71bde1d20f64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:04.931575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:04.931575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:04.931575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:04.931575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:04.931575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:04.931575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:04.931575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:04.931575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:04.931575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:04.931575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:04.931575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:04.931575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:04.931575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:04.931575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:04.933361 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:04.933361 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:04.933361 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:04.933361 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:04.933361 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:04.933361 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:04.933361 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:04.933361 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:04.933361 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:04.933361 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:04.933361 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:04.933361 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:04.933361 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:04.933361 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:04.933361 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:04.933361 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:04.933361 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:04.935127 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:04.935127 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:04.935127 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:04.935127 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:04.935127 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 288/580] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:04.935127 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a0225-8483-4d5a-b403-71bde1d20f64 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:04.970807 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:05.028207 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d398310-9d8d-40d0-bd99-d42a7a20710e None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:05.044787 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:05.113425 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c0880-af86-49bc-b333-726e3280a77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:05.114554 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c0880-af86-49bc-b333-726e3280a77b None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:05.115033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c0880-af86-49bc-b333-726e3280a77b None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:05.115518 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c0880-af86-49bc-b333-726e3280a77b None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:05.117185 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1c0880-af86-49bc-b333-726e3280a77b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:05.117743 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1c0880-af86-49bc-b333-726e3280a77b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:05.120622 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1c0880-af86-49bc-b333-726e3280a77b None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:05.161642 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 293/581] 10.222.0.45 () {56 vars in 1137 bytes} [Mon Aug 1 13:54:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:05.163544 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a0225-8483-4d5a-b403-71bde1d20f64 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:05.164311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c0880-af86-49bc-b333-726e3280a77b None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:05.247417 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacadedd-c69c-4cab-bb18-cd8ef62cdebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:05.248296 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacadedd-c69c-4cab-bb18-cd8ef62cdebc None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:05.248852 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacadedd-c69c-4cab-bb18-cd8ef62cdebc None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:05.249359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacadedd-c69c-4cab-bb18-cd8ef62cdebc None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:05.250917 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacadedd-c69c-4cab-bb18-cd8ef62cdebc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:05.251511 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacadedd-c69c-4cab-bb18-cd8ef62cdebc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:05.254425 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacadedd-c69c-4cab-bb18-cd8ef62cdebc None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:05.288903 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 289/582] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:05.291102 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacadedd-c69c-4cab-bb18-cd8ef62cdebc None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:05.406995 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c0880-af86-49bc-b333-726e3280a77b None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:05.476377 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:05.481763 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5efe14d-acea-4409-b8bf-dd7ebf3f309a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:05.482958 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efe14d-acea-4409-b8bf-dd7ebf3f309a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:05.483464 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efe14d-acea-4409-b8bf-dd7ebf3f309a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:05.484017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efe14d-acea-4409-b8bf-dd7ebf3f309a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:05.510547 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5efe14d-acea-4409-b8bf-dd7ebf3f309a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:05.513718 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5efe14d-acea-4409-b8bf-dd7ebf3f309a None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:05.543615 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacadedd-c69c-4cab-bb18-cd8ef62cdebc None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:05.556946 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-b5efe14d-acea-4409-b8bf-dd7ebf3f309a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:05.556946 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:05.556946 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:05.556946 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:05.556946 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:05.556946 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:05.556946 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:05.556946 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:05.556946 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:05.556946 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:05.556946 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:05.556946 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:05.556946 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:05.556946 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:05.556946 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:05.558763 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:05.558763 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:05.558763 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:05.558763 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:05.558763 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:05.558763 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:05.558763 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:05.558763 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:05.558763 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:05.558763 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:05.558763 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:05.558763 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:05.558763 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:05.558763 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:05.558763 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:05.558763 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:05.558763 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:05.560527 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:05.560527 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:05.560527 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:05.560527 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:05.565084 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 294/583] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:05.566403 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5efe14d-acea-4409-b8bf-dd7ebf3f309a None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:05.608681 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:05.633263 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e701eb-bf3a-4382-a487-73988e46d764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:05.634436 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e701eb-bf3a-4382-a487-73988e46d764 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:05.634957 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e701eb-bf3a-4382-a487-73988e46d764 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:05.635478 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e701eb-bf3a-4382-a487-73988e46d764 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:05.637002 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e701eb-bf3a-4382-a487-73988e46d764 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:05.637594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e701eb-bf3a-4382-a487-73988e46d764 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:05.640812 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e701eb-bf3a-4382-a487-73988e46d764 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:05.673252 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 290/584] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:05.673252 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e701eb-bf3a-4382-a487-73988e46d764 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:05.817941 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5efe14d-acea-4409-b8bf-dd7ebf3f309a None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:05.888618 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e701eb-bf3a-4382-a487-73988e46d764 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:05.895853 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b722d5-0ae9-4364-9ef4-4cd6572aab1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:05.897094 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b722d5-0ae9-4364-9ef4-4cd6572aab1e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:05.897608 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b722d5-0ae9-4364-9ef4-4cd6572aab1e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:05.898128 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b722d5-0ae9-4364-9ef4-4cd6572aab1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:05.921453 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b722d5-0ae9-4364-9ef4-4cd6572aab1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:05.924561 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b722d5-0ae9-4364-9ef4-4cd6572aab1e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:05.956895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-c1b722d5-0ae9-4364-9ef4-4cd6572aab1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:05.956895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:05.956895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:05.956895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:05.956895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:05.956895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:05.956895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:05.956895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:05.956895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:05.956895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:05.956895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:05.956895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:05.956895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:05.956895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:05.956895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:05.958624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:05.958624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:05.958624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:05.958624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:05.958624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:05.958624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:05.958624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:05.958624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:05.958624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:05.958624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:05.958624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:05.958624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:05.958624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:05.958624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:05.958624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:05.958624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:05.958624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:05.960465 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:05.960465 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:05.960465 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:05.960465 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:05.960465 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 295/585] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:05.961037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b722d5-0ae9-4364-9ef4-4cd6572aab1e None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:05.980649 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd6760-e54a-4be3-8a74-e611569738aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:05.981746 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd6760-e54a-4be3-8a74-e611569738aa None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:05.982214 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd6760-e54a-4be3-8a74-e611569738aa None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:05.982659 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd6760-e54a-4be3-8a74-e611569738aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:05.986458 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:05.994889 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfd6760-e54a-4be3-8a74-e611569738aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:05.996832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfd6760-e54a-4be3-8a74-e611569738aa None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:06.010658 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-9cfd6760-e54a-4be3-8a74-e611569738aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:06.010658 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:06.010658 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:06.010658 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:06.010658 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:06.010658 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:06.010658 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:06.010658 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:06.010658 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:06.010658 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:06.010658 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:06.010658 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:06.010658 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:54:06.010658 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:54:06.010658 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:54:06.012946 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:54:06.012946 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:06.012946 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:06.012946 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:06.012946 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:06.012946 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:06.012946 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:06.012946 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:06.012946 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:06.012946 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:06.012946 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:06.012946 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:06.012946 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:06.012946 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:06.012946 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:06.012946 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:06.012946 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:06.015060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:54:06.015060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:54:06.015060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:54:06.015060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:54:06.015060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:54:06.015060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:54:06.015060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:06.015060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:06.015060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 291/586] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:54:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:06.015060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd6760-e54a-4be3-8a74-e611569738aa None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:06.112218 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:06.171014 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b722d5-0ae9-4364-9ef4-4cd6572aab1e None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:06.203965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd6760-e54a-4be3-8a74-e611569738aa None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:06.223502 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1573b2db-ed2c-457f-86a4-d558436a5f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:06.224604 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573b2db-ed2c-457f-86a4-d558436a5f57 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:06.225093 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573b2db-ed2c-457f-86a4-d558436a5f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:06.225570 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573b2db-ed2c-457f-86a4-d558436a5f57 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:06.226842 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1573b2db-ed2c-457f-86a4-d558436a5f57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:06.227380 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1573b2db-ed2c-457f-86a4-d558436a5f57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:06.229997 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1573b2db-ed2c-457f-86a4-d558436a5f57 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:06.252978 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 296/587] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:06.255249 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31638af-6473-4a00-9afd-bbfb8da62b86 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:06.255754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31638af-6473-4a00-9afd-bbfb8da62b86 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:06.256234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31638af-6473-4a00-9afd-bbfb8da62b86 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:06.258752 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 297/588] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:06.260121 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31638af-6473-4a00-9afd-bbfb8da62b86 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:06.270356 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42069cc-a24f-4f7d-8c5b-cc34689b8cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:06.271436 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42069cc-a24f-4f7d-8c5b-cc34689b8cd7 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:06.271927 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42069cc-a24f-4f7d-8c5b-cc34689b8cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:06.272458 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42069cc-a24f-4f7d-8c5b-cc34689b8cd7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:06.286327 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42069cc-a24f-4f7d-8c5b-cc34689b8cd7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:06.289075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42069cc-a24f-4f7d-8c5b-cc34689b8cd7 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:06.312342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-a42069cc-a24f-4f7d-8c5b-cc34689b8cd7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 13:54:06.312342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:06.312342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:06.312342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:06.312342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:06.312342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:06.312342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:06.312342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:06.312342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:06.312342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:06.312342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:06.312342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:06.312342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:54:06.312342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:54:06.312342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:54:06.312342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:54:06.313919 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:06.313919 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:06.313919 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:54:06.313919 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:06.313919 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:54:06.313919 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:06.313919 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:06.313919 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:06.313919 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:06.313919 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:06.313919 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:06.313919 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:06.313919 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:06.313919 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:06.313919 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:06.313919 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:06.313919 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:06.313919 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:06.315473 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:06.315473 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:06.315473 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:54:06.315473 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:54:06.315473 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:54:06.315473 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:54:06.315473 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:54:06.315473 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:54:06.315473 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 13:54:06.315473 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:06.315473 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 292/589] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:54:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:06.316531 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42069cc-a24f-4f7d-8c5b-cc34689b8cd7 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:06.419858 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:06.474208 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31638af-6473-4a00-9afd-bbfb8da62b86 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:06.508088 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42069cc-a24f-4f7d-8c5b-cc34689b8cd7 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:06.524508 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:06.537974 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36e70b-5a7f-4246-b7d0-5f25cfa523bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:06.538932 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36e70b-5a7f-4246-b7d0-5f25cfa523bc None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:06.544666 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36e70b-5a7f-4246-b7d0-5f25cfa523bc None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:06.544666 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36e70b-5a7f-4246-b7d0-5f25cfa523bc None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:06.544666 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36e70b-5a7f-4246-b7d0-5f25cfa523bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:06.544666 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36e70b-5a7f-4246-b7d0-5f25cfa523bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:06.544666 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36e70b-5a7f-4246-b7d0-5f25cfa523bc None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:06.563604 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 298/590] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:06.564180 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36e70b-5a7f-4246-b7d0-5f25cfa523bc None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:06.595797 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fef623-0e1d-4eb4-878f-095142d13ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:06.596879 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fef623-0e1d-4eb4-878f-095142d13ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:06.597364 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fef623-0e1d-4eb4-878f-095142d13ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:06.597846 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fef623-0e1d-4eb4-878f-095142d13ec8 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:06.599249 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fef623-0e1d-4eb4-878f-095142d13ec8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:06.599788 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fef623-0e1d-4eb4-878f-095142d13ec8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:06.602535 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fef623-0e1d-4eb4-878f-095142d13ec8 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:06.624860 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 293/591] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:54:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:06.626858 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d358d-a178-4b2c-ac2f-34996db56706 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:06.627347 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d358d-a178-4b2c-ac2f-34996db56706 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:06.627831 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d358d-a178-4b2c-ac2f-34996db56706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:06.658654 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d53d358d-a178-4b2c-ac2f-34996db56706 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:06.660704 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d53d358d-a178-4b2c-ac2f-34996db56706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:06.731540 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36e70b-5a7f-4246-b7d0-5f25cfa523bc None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:06.779681 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 294/592] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:54:06.780707 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d358d-a178-4b2c-ac2f-34996db56706 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:06.805748 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d8868b-0c0c-4407-a0c5-e158e14bea36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:06.807522 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8868b-0c0c-4407-a0c5-e158e14bea36 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:06.808202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8868b-0c0c-4407-a0c5-e158e14bea36 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:06.809625 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8868b-0c0c-4407-a0c5-e158e14bea36 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:06.825125 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d8868b-0c0c-4407-a0c5-e158e14bea36 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:06.827439 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d8868b-0c0c-4407-a0c5-e158e14bea36 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:06.845661 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-01d8868b-0c0c-4407-a0c5-e158e14bea36 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 13:54:06.845661 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:06.845661 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:06.845661 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:06.845661 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:06.845661 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:06.845661 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:06.845661 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:06.845661 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:06.845661 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:06.845661 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:06.845661 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:06.845661 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:54:06.845661 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:54:06.845661 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:54:06.845661 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:54:06.847379 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:06.847379 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:06.847379 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:54:06.847379 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:06.847379 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:54:06.847379 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:06.847379 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:06.847379 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:06.847379 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:06.847379 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:06.847379 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:06.847379 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:06.847379 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:06.847379 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:06.847379 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:06.847379 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:06.847379 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:06.847379 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:06.849123 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:06.849123 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:06.849123 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:54:06.849123 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:54:06.849123 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:54:06.849123 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:54:06.849123 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:54:06.849123 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:54:06.849123 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 13:54:06.849123 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:06.849123 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 299/593] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 13:54:06 2022] GET /identity/v3/users/neutron => generated 85 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:06.850223 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d8868b-0c0c-4407-a0c5-e158e14bea36 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:06.870714 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:06.918659 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:06.964981 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d358d-a178-4b2c-ac2f-34996db56706 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:07.012325 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d8868b-0c0c-4407-a0c5-e158e14bea36 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:07.015188 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f7823-579e-414f-8629-b608c9c64d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:07.016224 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f7823-579e-414f-8629-b608c9c64d22 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:07.016770 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f7823-579e-414f-8629-b608c9c64d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:07.017199 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f7823-579e-414f-8629-b608c9c64d22 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:07.018693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f7823-579e-414f-8629-b608c9c64d22 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:07.019552 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f7823-579e-414f-8629-b608c9c64d22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:07.022065 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f7823-579e-414f-8629-b608c9c64d22 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:07.056449 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 295/594] 10.222.0.45 () {56 vars in 1137 bytes} [Mon Aug 1 13:54:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:07.057943 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f7823-579e-414f-8629-b608c9c64d22 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:07.073080 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9296da7-8360-407b-9455-d24c84f6db52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:07.074066 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9296da7-8360-407b-9455-d24c84f6db52 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:07.074485 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9296da7-8360-407b-9455-d24c84f6db52 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:07.074903 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9296da7-8360-407b-9455-d24c84f6db52 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:07.087238 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9296da7-8360-407b-9455-d24c84f6db52 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:07.090044 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9296da7-8360-407b-9455-d24c84f6db52 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:07.113855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-e9296da7-8360-407b-9455-d24c84f6db52 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:07.113855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:07.113855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:07.113855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:07.113855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:07.113855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:07.113855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:07.113855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:07.113855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:07.113855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:07.113855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:07.113855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:07.113855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:54:07.113855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:54:07.113855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:54:07.115057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:54:07.115057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:07.115057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:07.115057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:07.115057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:07.115057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:07.115057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:07.115057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:07.115057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:07.115057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:07.115057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:07.115057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:07.115057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:07.115057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:07.115057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:07.115057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:07.115057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:07.116795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:54:07.116795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:54:07.116795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:54:07.116795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:54:07.116795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:54:07.116795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:54:07.116795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:07.116795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:07.116795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 300/595] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:54:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:07.117714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9296da7-8360-407b-9455-d24c84f6db52 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:07.227952 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f7823-579e-414f-8629-b608c9c64d22 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:07.231060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:07.290621 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db67d24-de8c-462d-903f-f19b65be8f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:07.291533 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db67d24-de8c-462d-903f-f19b65be8f60 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:07.291952 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db67d24-de8c-462d-903f-f19b65be8f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:07.292428 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db67d24-de8c-462d-903f-f19b65be8f60 None admin] PATH_INFO: `/v3/projects/36f063d8f65d494a89dbbd72c53417c8/users/5b3adc14dcaf4ac79b43bb97fa1ba1d4/roles/d17972f7fe97462b8f9f08aed9bc9cb5` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:07.300769 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9296da7-8360-407b-9455-d24c84f6db52 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:07.316039 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:07.319933 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db67d24-de8c-462d-903f-f19b65be8f60 None admin] RBAC: Authorizing `identity:create_grant(project_id=36f063d8f65d494a89dbbd72c53417c8, user_id=5b3adc14dcaf4ac79b43bb97fa1ba1d4, role_id=d17972f7fe97462b8f9f08aed9bc9cb5)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:07.322885 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db67d24-de8c-462d-903f-f19b65be8f60 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:07.345847 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 296/596] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 13:54:07 2022] PUT /identity/v3/projects/36f063d8f65d494a89dbbd72c53417c8/users/5b3adc14dcaf4ac79b43bb97fa1ba1d4/roles/d17972f7fe97462b8f9f08aed9bc9cb5 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:54:07.346537 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db67d24-de8c-462d-903f-f19b65be8f60 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:07.354537 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3210c-6995-4a46-9902-ce72436cb280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:07.355579 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3210c-6995-4a46-9902-ce72436cb280 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:07.356034 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3210c-6995-4a46-9902-ce72436cb280 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:07.357368 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3210c-6995-4a46-9902-ce72436cb280 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:07.375805 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b3210c-6995-4a46-9902-ce72436cb280 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:54:07.386884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-14b3210c-6995-4a46-9902-ce72436cb280 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:54:07.386884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:07.386884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:07.386884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:07.386884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:07.386884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:07.386884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:07.386884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:07.386884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:07.386884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:07.386884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:07.386884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:07.386884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:54:07.386884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:54:07.386884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:54:07.386884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:54:07.388799 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:54:07.388799 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:54:07.388799 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:54:07.388799 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:54:07.388799 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:07.388799 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:07.388799 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:07.388799 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:07.388799 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:07.388799 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:07.388799 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:07.388799 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:07.388799 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:07.388799 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:07.388799 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:07.388799 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:07.390290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:07.390290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:07.390290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:07.390290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:07.390290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:54:07.390290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:54:07.390290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:54:07.390290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:54:07.390290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:54:07.390290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:54:07.390290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:54:07.390290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:07.390290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 301/597] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 13:54:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:07.391653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3210c-6995-4a46-9902-ce72436cb280 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:07.562334 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db67d24-de8c-462d-903f-f19b65be8f60 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:07.585454 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3210c-6995-4a46-9902-ce72436cb280 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:07.633323 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b2205e-e459-468d-a7a8-bcf676da4bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:07.634208 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2205e-e459-468d-a7a8-bcf676da4bca None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:07.634361 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2205e-e459-468d-a7a8-bcf676da4bca None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:07.634644 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2205e-e459-468d-a7a8-bcf676da4bca None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:07.640494 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b2205e-e459-468d-a7a8-bcf676da4bca None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:07.640494 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b2205e-e459-468d-a7a8-bcf676da4bca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:07.640494 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b2205e-e459-468d-a7a8-bcf676da4bca None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:07.657107 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02965f07-bf08-4d8a-b662-77362faf7852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:07.658331 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02965f07-bf08-4d8a-b662-77362faf7852 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:07.658902 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02965f07-bf08-4d8a-b662-77362faf7852 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:07.659820 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02965f07-bf08-4d8a-b662-77362faf7852 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:07.667876 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 297/598] 10.222.0.45 () {56 vars in 1139 bytes} [Mon Aug 1 13:54:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:07.670051 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b2205e-e459-468d-a7a8-bcf676da4bca None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:07.681947 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02965f07-bf08-4d8a-b662-77362faf7852 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:07.684636 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02965f07-bf08-4d8a-b662-77362faf7852 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:07.698107 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:07.715113 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-02965f07-bf08-4d8a-b662-77362faf7852 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:07.715113 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:07.715113 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:07.715113 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:07.715113 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:07.715113 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:07.715113 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:07.715113 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:07.715113 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:07.715113 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:07.715113 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:07.715113 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:07.715113 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:07.715113 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:07.715113 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:07.717061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:07.717061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:07.717061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:07.717061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:07.717061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:07.717061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:07.717061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:07.717061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:07.717061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:07.717061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:07.717061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:07.717061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:07.717061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:07.717061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:07.717061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:07.717061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:07.717061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:07.719772 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:07.719772 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:07.719772 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:07.719772 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:07.725105 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 302/599] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:07.726077 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02965f07-bf08-4d8a-b662-77362faf7852 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:07.765405 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:07.865831 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b2205e-e459-468d-a7a8-bcf676da4bca None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:07.916996 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02965f07-bf08-4d8a-b662-77362faf7852 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:07.931848 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b861fb9-afa8-42f8-ab0c-b594f0932c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:07.932918 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b861fb9-afa8-42f8-ab0c-b594f0932c5d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:07.933355 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b861fb9-afa8-42f8-ab0c-b594f0932c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:07.933822 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b861fb9-afa8-42f8-ab0c-b594f0932c5d None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:07.935269 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b861fb9-afa8-42f8-ab0c-b594f0932c5d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:07.935782 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b861fb9-afa8-42f8-ab0c-b594f0932c5d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:07.938680 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b861fb9-afa8-42f8-ab0c-b594f0932c5d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:07.967956 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 298/600] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:54:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:07.967956 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b861fb9-afa8-42f8-ab0c-b594f0932c5d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:07.980944 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e1187-9760-467c-9e9f-0cfb7c9c7277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:07.981592 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e1187-9760-467c-9e9f-0cfb7c9c7277 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:07.981875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e1187-9760-467c-9e9f-0cfb7c9c7277 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:07.982138 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e1187-9760-467c-9e9f-0cfb7c9c7277 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:07.982885 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377e1187-9760-467c-9e9f-0cfb7c9c7277 None admin] RBAC: Adding query filter params (name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:07.983189 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377e1187-9760-467c-9e9f-0cfb7c9c7277 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:07.984665 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377e1187-9760-467c-9e9f-0cfb7c9c7277 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:07.999358 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 303/601] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 13:54:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:07.999358 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e1187-9760-467c-9e9f-0cfb7c9c7277 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:08.093243 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:08.152192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:08.165499 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b861fb9-afa8-42f8-ab0c-b594f0932c5d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:08.189019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e1187-9760-467c-9e9f-0cfb7c9c7277 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:08.227540 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab53439-6cc9-443a-b38c-29eded4c3ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:08.228416 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab53439-6cc9-443a-b38c-29eded4c3ebf None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:08.228690 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab53439-6cc9-443a-b38c-29eded4c3ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:08.228965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab53439-6cc9-443a-b38c-29eded4c3ebf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:08.250352 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5d2f0-9b95-46cd-8c22-cc687f650986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:08.251672 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5d2f0-9b95-46cd-8c22-cc687f650986 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:08.251672 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5d2f0-9b95-46cd-8c22-cc687f650986 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:08.251993 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5d2f0-9b95-46cd-8c22-cc687f650986 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:08.252772 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5d2f0-9b95-46cd-8c22-cc687f650986 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:08.253203 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5d2f0-9b95-46cd-8c22-cc687f650986 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:08.255486 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5d2f0-9b95-46cd-8c22-cc687f650986 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:08.255711 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab53439-6cc9-443a-b38c-29eded4c3ebf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:08.258028 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab53439-6cc9-443a-b38c-29eded4c3ebf None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:08.277901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 304/602] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:08.279607 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5d2f0-9b95-46cd-8c22-cc687f650986 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:08.282834 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-3ab53439-6cc9-443a-b38c-29eded4c3ebf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:08.282834 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:08.282834 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:08.282834 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:08.282834 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:08.282834 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:08.282834 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:08.282834 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:08.282834 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:08.282834 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:08.282834 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:08.282834 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:08.282834 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:08.282834 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:08.282834 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:08.284510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:08.284510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:08.284510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:08.284510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:08.284510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:08.284510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:08.284510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:08.284510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:08.284510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:08.284510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:08.284510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:08.284510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:08.284510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:08.284510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:08.284510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:08.284510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:08.284510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:08.286191 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:08.286191 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:08.286191 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:08.286191 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:08.288718 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 299/603] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:08.290411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab53439-6cc9-443a-b38c-29eded4c3ebf None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:08.490429 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5d2f0-9b95-46cd-8c22-cc687f650986 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:08.493590 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab53439-6cc9-443a-b38c-29eded4c3ebf None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:08.546062 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:08.572720 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18bb16e-2de1-4545-bd36-9f0ef15348a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:08.573675 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18bb16e-2de1-4545-bd36-9f0ef15348a6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:08.574115 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18bb16e-2de1-4545-bd36-9f0ef15348a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:08.574563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18bb16e-2de1-4545-bd36-9f0ef15348a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:08.575579 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227aea92-2393-4444-ab63-75e20a178a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:08.576813 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227aea92-2393-4444-ab63-75e20a178a9b None admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:08.576813 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227aea92-2393-4444-ab63-75e20a178a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:08.578417 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227aea92-2393-4444-ab63-75e20a178a9b None admin] PATH_INFO: `/v3/projects/36f063d8f65d494a89dbbd72c53417c8/users/8f4c3d46ac7243aeb722a4410f40d846/roles/d17972f7fe97462b8f9f08aed9bc9cb5` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:08.610538 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:08.612812 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18bb16e-2de1-4545-bd36-9f0ef15348a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:08.615079 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18bb16e-2de1-4545-bd36-9f0ef15348a6 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:08.617281 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227aea92-2393-4444-ab63-75e20a178a9b None admin] RBAC: Authorizing `identity:create_grant(project_id=36f063d8f65d494a89dbbd72c53417c8, user_id=8f4c3d46ac7243aeb722a4410f40d846, role_id=d17972f7fe97462b8f9f08aed9bc9cb5)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:08.620932 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227aea92-2393-4444-ab63-75e20a178a9b None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:08.640433 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-f18bb16e-2de1-4545-bd36-9f0ef15348a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:08.640433 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:08.640433 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:08.640433 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:08.640433 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:08.640433 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:08.640433 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:08.640433 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:08.640433 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:08.640433 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:08.640433 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:08.640433 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:08.640433 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:08.640433 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:08.640433 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:08.648794 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:08.648794 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:08.648794 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:08.648794 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:08.648794 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:08.648794 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:08.648794 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:08.648794 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:08.648794 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:08.648794 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:08.648794 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:08.648794 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:08.648794 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:08.648794 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:08.648794 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:08.648794 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:08.648794 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:08.650962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:08.650962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:08.650962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:08.650962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:08.650962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 300/604] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:08.650962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18bb16e-2de1-4545-bd36-9f0ef15348a6 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:08.651775 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 305/605] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 13:54:08 2022] PUT /identity/v3/projects/36f063d8f65d494a89dbbd72c53417c8/users/8f4c3d46ac7243aeb722a4410f40d846/roles/d17972f7fe97462b8f9f08aed9bc9cb5 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:54:08.651775 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227aea92-2393-4444-ab63-75e20a178a9b None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:08.854570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18bb16e-2de1-4545-bd36-9f0ef15348a6 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:08.897065 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227aea92-2393-4444-ab63-75e20a178a9b None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:08.923907 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02379d83-ffa6-4d16-a4e8-df56694401dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:08.925069 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02379d83-ffa6-4d16-a4e8-df56694401dd None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:08.925534 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02379d83-ffa6-4d16-a4e8-df56694401dd None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:08.926007 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02379d83-ffa6-4d16-a4e8-df56694401dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:08.941797 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02379d83-ffa6-4d16-a4e8-df56694401dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:08.944627 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02379d83-ffa6-4d16-a4e8-df56694401dd None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:08.969504 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2565e211-9042-4774-9015-66f1202e30e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:08.970152 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565e211-9042-4774-9015-66f1202e30e3 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:08.970275 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-02379d83-ffa6-4d16-a4e8-df56694401dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:08.970275 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:08.970275 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:08.970275 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:08.970275 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:08.970275 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:08.970275 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:08.970275 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:08.970275 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:08.970275 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:08.970275 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:08.970275 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:08.970275 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:54:08.970275 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:54:08.970275 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:54:08.971108 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:54:08.971108 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:08.971108 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:08.971108 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:08.971108 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:08.971108 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:08.971108 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:08.971108 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:08.971108 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:08.971108 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:08.971108 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:08.971108 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:08.971108 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:08.971108 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:08.971108 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:08.971108 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:08.971108 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:08.971999 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:54:08.971999 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:54:08.971999 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:54:08.971999 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:54:08.971999 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:54:08.971999 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:54:08.971999 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:08.971999 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:08.972474 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565e211-9042-4774-9015-66f1202e30e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:08.972474 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565e211-9042-4774-9015-66f1202e30e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:08.972474 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2565e211-9042-4774-9015-66f1202e30e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:08.972474 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2565e211-9042-4774-9015-66f1202e30e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:08.973259 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 301/606] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:54:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:08.973360 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2565e211-9042-4774-9015-66f1202e30e3 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:08.974555 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02379d83-ffa6-4d16-a4e8-df56694401dd None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:08.993946 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 306/607] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:08.995656 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2565e211-9042-4774-9015-66f1202e30e3 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:09.066452 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:09.070621 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:09.127742 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2565e211-9042-4774-9015-66f1202e30e3 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:09.166884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d89827e-c253-4f66-ac61-fe1631963d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:09.167522 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d89827e-c253-4f66-ac61-fe1631963d16 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:09.167843 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d89827e-c253-4f66-ac61-fe1631963d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:09.168607 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d89827e-c253-4f66-ac61-fe1631963d16 None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:09.170018 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d89827e-c253-4f66-ac61-fe1631963d16 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:09.170533 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d89827e-c253-4f66-ac61-fe1631963d16 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:09.173413 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d89827e-c253-4f66-ac61-fe1631963d16 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:09.196917 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02379d83-ffa6-4d16-a4e8-df56694401dd None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:09.204336 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 307/608] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:54:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:09.204336 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d89827e-c253-4f66-ac61-fe1631963d16 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:09.278245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9013b-4def-4edb-acd8-9fec815f7a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:09.279808 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9013b-4def-4edb-acd8-9fec815f7a29 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:09.281030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9013b-4def-4edb-acd8-9fec815f7a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:09.281798 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9013b-4def-4edb-acd8-9fec815f7a29 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:09.283614 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9013b-4def-4edb-acd8-9fec815f7a29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:09.284586 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9013b-4def-4edb-acd8-9fec815f7a29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:09.287566 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9013b-4def-4edb-acd8-9fec815f7a29 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:09.320870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 302/609] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:09.322996 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9013b-4def-4edb-acd8-9fec815f7a29 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:09.407828 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d89827e-c253-4f66-ac61-fe1631963d16 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:09.436788 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:09.480556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ced6b6-dd05-48a4-9bd3-c48e6f7a13be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:09.481678 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ced6b6-dd05-48a4-9bd3-c48e6f7a13be None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:09.482234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ced6b6-dd05-48a4-9bd3-c48e6f7a13be None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:09.482794 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ced6b6-dd05-48a4-9bd3-c48e6f7a13be None admin] PATH_INFO: `/v3/projects/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:09.496833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ced6b6-dd05-48a4-9bd3-c48e6f7a13be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:09.499968 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ced6b6-dd05-48a4-9bd3-c48e6f7a13be None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:09.524213 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-e6ced6b6-dd05-48a4-9bd3-c48e6f7a13be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:09.524213 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:09.524213 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:09.524213 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:09.524213 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:09.524213 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:09.524213 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:09.524213 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:09.524213 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:09.524213 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:09.524213 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:09.524213 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:09.524213 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:54:09.524213 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:54:09.524213 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:54:09.526008 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:54:09.526008 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:09.526008 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:09.526008 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:09.526008 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:09.526008 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:09.526008 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:09.526008 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:09.526008 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:09.526008 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:09.526008 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:09.526008 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:09.526008 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:09.526008 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:09.526008 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:09.526008 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:09.526008 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:09.527780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:54:09.527780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:54:09.527780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:54:09.527780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:54:09.527780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:54:09.527780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:54:09.527780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:09.527780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:09.530933 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:09.546771 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9013b-4def-4edb-acd8-9fec815f7a29 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:09.553324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 308/610] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:54:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:09.554440 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ced6b6-dd05-48a4-9bd3-c48e6f7a13be None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:09.618557 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70769eb-c8fb-4f24-ad3c-770b35f344e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:09.620093 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70769eb-c8fb-4f24-ad3c-770b35f344e9 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:09.620847 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70769eb-c8fb-4f24-ad3c-770b35f344e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:09.621691 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70769eb-c8fb-4f24-ad3c-770b35f344e9 None admin] PATH_INFO: `/v3/projects/36f063d8f65d494a89dbbd72c53417c8/users/e7317c8829cb4fa1b30711f1db575cab/roles/d17972f7fe97462b8f9f08aed9bc9cb5` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:09.649447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70769eb-c8fb-4f24-ad3c-770b35f344e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=36f063d8f65d494a89dbbd72c53417c8, user_id=e7317c8829cb4fa1b30711f1db575cab, role_id=d17972f7fe97462b8f9f08aed9bc9cb5)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:09.652534 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70769eb-c8fb-4f24-ad3c-770b35f344e9 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:09.672341 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 303/611] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 13:54:09 2022] PUT /identity/v3/projects/36f063d8f65d494a89dbbd72c53417c8/users/e7317c8829cb4fa1b30711f1db575cab/roles/d17972f7fe97462b8f9f08aed9bc9cb5 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:54:09.681693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70769eb-c8fb-4f24-ad3c-770b35f344e9 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:09.760970 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ced6b6-dd05-48a4-9bd3-c48e6f7a13be None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:09.844031 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd2378-cdac-4f25-8b74-bab82c88dab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:09.844031 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd2378-cdac-4f25-8b74-bab82c88dab7 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:09.844031 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd2378-cdac-4f25-8b74-bab82c88dab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:09.844031 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd2378-cdac-4f25-8b74-bab82c88dab7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:09.875315 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd2378-cdac-4f25-8b74-bab82c88dab7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:09.880287 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd2378-cdac-4f25-8b74-bab82c88dab7 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:09.921243 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-15dd2378-cdac-4f25-8b74-bab82c88dab7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 13:54:09.921243 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:09.921243 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:09.921243 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:09.921243 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:09.921243 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:09.921243 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:09.921243 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:09.921243 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:09.921243 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:09.921243 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:09.921243 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:09.921243 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:54:09.921243 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:54:09.921243 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:54:09.921243 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:54:09.923164 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:09.923164 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:09.923164 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:54:09.923164 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:09.923164 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:54:09.923164 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:09.923164 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:09.923164 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:09.923164 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:09.923164 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:09.923164 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:09.923164 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:09.923164 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:09.923164 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:09.923164 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:09.923164 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:09.923164 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:09.923164 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:09.924484 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:09.924484 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:09.924484 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:54:09.924484 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:54:09.924484 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:54:09.924484 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:54:09.924484 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:54:09.924484 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:54:09.924484 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 13:54:09.924484 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:09.928320 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 309/612] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:54:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:09.934279 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd2378-cdac-4f25-8b74-bab82c88dab7 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:10.046819 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70769eb-c8fb-4f24-ad3c-770b35f344e9 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:10.049912 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:10.111508 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359e659-03f3-4111-8589-a0cd89e4cf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:10.112512 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359e659-03f3-4111-8589-a0cd89e4cf24 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:10.112980 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359e659-03f3-4111-8589-a0cd89e4cf24 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:10.113473 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359e659-03f3-4111-8589-a0cd89e4cf24 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:10.114873 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6359e659-03f3-4111-8589-a0cd89e4cf24 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:10.115220 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6359e659-03f3-4111-8589-a0cd89e4cf24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:10.119043 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6359e659-03f3-4111-8589-a0cd89e4cf24 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:10.142275 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 304/613] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:54:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 464 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:10.143796 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359e659-03f3-4111-8589-a0cd89e4cf24 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:10.149253 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:10.179608 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd2378-cdac-4f25-8b74-bab82c88dab7 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:10.254017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c9860-1c93-412d-90c5-8a88d2ded397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:10.257237 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c9860-1c93-412d-90c5-8a88d2ded397 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:10.257720 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c9860-1c93-412d-90c5-8a88d2ded397 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:10.258149 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c9860-1c93-412d-90c5-8a88d2ded397 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:10.260798 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c9860-1c93-412d-90c5-8a88d2ded397 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:10.261293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c9860-1c93-412d-90c5-8a88d2ded397 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:10.264443 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c9860-1c93-412d-90c5-8a88d2ded397 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:10.294070 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 310/614] 10.222.0.45 () {56 vars in 1135 bytes} [Mon Aug 1 13:54:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:54:10.302907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c9860-1c93-412d-90c5-8a88d2ded397 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:10.377459 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359e659-03f3-4111-8589-a0cd89e4cf24 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:10.452487 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d46f52-cb71-485b-a237-6db28f661993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:10.453627 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d46f52-cb71-485b-a237-6db28f661993 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:10.454094 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d46f52-cb71-485b-a237-6db28f661993 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:10.454544 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d46f52-cb71-485b-a237-6db28f661993 None admin] PATH_INFO: `/v3/projects/36f063d8f65d494a89dbbd72c53417c8/users/6dd44b7a78594288b6aabb393fd6324a/roles/d17972f7fe97462b8f9f08aed9bc9cb5` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:10.500876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:10.511574 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d46f52-cb71-485b-a237-6db28f661993 None admin] RBAC: Authorizing `identity:create_grant(project_id=36f063d8f65d494a89dbbd72c53417c8, user_id=6dd44b7a78594288b6aabb393fd6324a, role_id=d17972f7fe97462b8f9f08aed9bc9cb5)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:10.514259 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d46f52-cb71-485b-a237-6db28f661993 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:10.523942 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c9860-1c93-412d-90c5-8a88d2ded397 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:10.564143 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 305/615] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 13:54:10 2022] PUT /identity/v3/projects/36f063d8f65d494a89dbbd72c53417c8/users/6dd44b7a78594288b6aabb393fd6324a/roles/d17972f7fe97462b8f9f08aed9bc9cb5 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 13:54:10.570610 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea7f09-85d6-4f2b-ac8f-3a7b274c2dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:10.571895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea7f09-85d6-4f2b-ac8f-3a7b274c2dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:10.572508 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea7f09-85d6-4f2b-ac8f-3a7b274c2dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:10.573015 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea7f09-85d6-4f2b-ac8f-3a7b274c2dc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:10.586408 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ea7f09-85d6-4f2b-ac8f-3a7b274c2dc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:10.588514 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ea7f09-85d6-4f2b-ac8f-3a7b274c2dc3 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:10.607109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-e3ea7f09-85d6-4f2b-ac8f-3a7b274c2dc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:10.607109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:10.607109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:10.607109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:10.607109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:10.607109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:10.607109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:10.607109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:10.607109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:10.607109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:10.607109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:10.607109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:10.607109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:10.607109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:10.607109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:10.608129 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:10.608129 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:10.608129 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:10.608129 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:10.608129 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:10.608129 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:10.608129 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:10.608129 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:10.608129 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:10.608129 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:10.608129 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:10.608129 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:10.608129 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:10.608129 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:10.608129 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:10.608129 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:10.608129 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:10.609380 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:10.609380 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:10.609380 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:10.609380 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:10.612119 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 311/616] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:10.617948 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b9857-38e5-4aa6-a883-21cb429bd852 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:10.618345 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b9857-38e5-4aa6-a883-21cb429bd852 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:10.618622 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b9857-38e5-4aa6-a883-21cb429bd852 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:10.619486 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea7f09-85d6-4f2b-ac8f-3a7b274c2dc3 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:10.620499 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 306/617] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:10.626571 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068e86a-e2bf-4dd5-9dd1-6cb5eb1e1d8d None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:10.626913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068e86a-e2bf-4dd5-9dd1-6cb5eb1e1d8d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:10.627125 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068e86a-e2bf-4dd5-9dd1-6cb5eb1e1d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:10.649101 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-7068e86a-e2bf-4dd5-9dd1-6cb5eb1e1d8d None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:10.650451 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7068e86a-e2bf-4dd5-9dd1-6cb5eb1e1d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:10.740020 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:10.772502 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea7f09-85d6-4f2b-ac8f-3a7b274c2dc3 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:10.780852 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 307/618] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:54:10.824427 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea714be5-2c64-4db6-8c26-58603e0877db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:10.825856 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068e86a-e2bf-4dd5-9dd1-6cb5eb1e1d8d None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:10.826303 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea714be5-2c64-4db6-8c26-58603e0877db None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:10.827045 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea714be5-2c64-4db6-8c26-58603e0877db None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:10.827840 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea714be5-2c64-4db6-8c26-58603e0877db None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:10.829529 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea714be5-2c64-4db6-8c26-58603e0877db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:10.830416 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea714be5-2c64-4db6-8c26-58603e0877db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:10.833452 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea714be5-2c64-4db6-8c26-58603e0877db None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:10.857270 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 312/619] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:54:10.865510 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea714be5-2c64-4db6-8c26-58603e0877db None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:10.872909 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:11.065901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea714be5-2c64-4db6-8c26-58603e0877db None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:11.081215 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068e86a-e2bf-4dd5-9dd1-6cb5eb1e1d8d None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:11.111686 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66163d29-a0b7-490b-982e-6c8c8e6901c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:11.112568 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66163d29-a0b7-490b-982e-6c8c8e6901c1 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:11.112791 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66163d29-a0b7-490b-982e-6c8c8e6901c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:11.112980 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66163d29-a0b7-490b-982e-6c8c8e6901c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:11.120992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66163d29-a0b7-490b-982e-6c8c8e6901c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:11.122580 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66163d29-a0b7-490b-982e-6c8c8e6901c1 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:11.136033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-66163d29-a0b7-490b-982e-6c8c8e6901c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:11.136033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:11.136033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:11.136033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:11.136033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:11.136033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:11.136033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:11.136033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:11.136033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:11.136033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:11.136033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:11.136033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:11.136033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:54:11.136033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:54:11.136033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:54:11.138266 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:54:11.138266 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:11.138266 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:11.138266 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:11.138266 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:11.138266 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:11.138266 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:11.138266 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:11.138266 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:11.138266 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:11.138266 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:11.138266 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:11.138266 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:11.138266 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:11.138266 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:11.138266 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:11.138266 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:11.140362 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:54:11.140362 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:54:11.140362 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:54:11.140362 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:54:11.140362 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:54:11.140362 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:54:11.140362 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:11.140362 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:11.140362 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 313/620] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:54:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:11.152159 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66163d29-a0b7-490b-982e-6c8c8e6901c1 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:11.174466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbcc1a7-707d-461d-972c-320ba1bee829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:11.175210 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcc1a7-707d-461d-972c-320ba1bee829 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:11.175493 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcc1a7-707d-461d-972c-320ba1bee829 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:11.175759 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcc1a7-707d-461d-972c-320ba1bee829 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:11.198038 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbcc1a7-707d-461d-972c-320ba1bee829 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:54:11.204580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-cdbcc1a7-707d-461d-972c-320ba1bee829 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:54:11.204580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:11.204580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:11.204580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:11.204580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:11.204580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:11.204580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:11.204580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:11.204580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:11.204580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:11.204580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:11.204580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:11.204580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:54:11.204580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:54:11.204580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:54:11.204580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:54:11.206518 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:54:11.206518 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:54:11.206518 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:54:11.206518 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:54:11.206518 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:11.206518 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:11.206518 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:11.206518 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:11.206518 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:11.206518 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:11.206518 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:11.206518 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:11.206518 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:11.206518 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:11.206518 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:11.206518 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:11.208221 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:11.208221 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:11.208221 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:11.208221 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:11.208221 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:54:11.208221 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:54:11.208221 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:54:11.208221 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:54:11.208221 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:54:11.208221 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:54:11.208221 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:54:11.208221 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:11.208221 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 308/621] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 13:54:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:11.212206 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbcc1a7-707d-461d-972c-320ba1bee829 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:11.294831 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:11.375282 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:11.406551 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbcc1a7-707d-461d-972c-320ba1bee829 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:11.475540 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69bf14-4e14-4f50-849a-4b3cf487750f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:11.476712 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69bf14-4e14-4f50-849a-4b3cf487750f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:11.477168 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69bf14-4e14-4f50-849a-4b3cf487750f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:11.477736 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69bf14-4e14-4f50-849a-4b3cf487750f None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:11.484320 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69bf14-4e14-4f50-849a-4b3cf487750f None admin] RBAC: Adding query filter params (name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:11.484320 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69bf14-4e14-4f50-849a-4b3cf487750f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:11.484320 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69bf14-4e14-4f50-849a-4b3cf487750f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:11.504653 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 309/622] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 13:54:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:54:11.509235 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69bf14-4e14-4f50-849a-4b3cf487750f None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:11.583432 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66163d29-a0b7-490b-982e-6c8c8e6901c1 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:11.721214 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69bf14-4e14-4f50-849a-4b3cf487750f None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:11.741440 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77136be1-f137-48a5-b595-61587d10efff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:11.742532 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77136be1-f137-48a5-b595-61587d10efff None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:11.743015 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77136be1-f137-48a5-b595-61587d10efff None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:11.743507 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77136be1-f137-48a5-b595-61587d10efff None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:11.744991 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77136be1-f137-48a5-b595-61587d10efff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:11.745542 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77136be1-f137-48a5-b595-61587d10efff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:11.748314 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77136be1-f137-48a5-b595-61587d10efff None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:11.781500 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:11.786070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947f88bc-0785-46ab-a73f-168f93fcd262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:11.787106 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f88bc-0785-46ab-a73f-168f93fcd262 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:11.787549 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f88bc-0785-46ab-a73f-168f93fcd262 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:11.787996 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f88bc-0785-46ab-a73f-168f93fcd262 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:11.808656 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947f88bc-0785-46ab-a73f-168f93fcd262 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:11.811545 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947f88bc-0785-46ab-a73f-168f93fcd262 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:11.812247 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 314/623] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:54:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 666 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:11.821619 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77136be1-f137-48a5-b595-61587d10efff None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:11.838796 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-947f88bc-0785-46ab-a73f-168f93fcd262 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:11.838796 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:11.838796 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:11.838796 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:11.838796 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:11.838796 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:11.838796 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:11.838796 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:11.838796 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:11.838796 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:11.838796 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:11.838796 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:11.838796 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:11.838796 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:11.838796 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:11.840648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:11.840648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:11.840648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:11.840648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:11.840648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:11.840648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:11.840648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:11.840648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:11.840648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:11.840648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:11.840648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:11.840648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:11.840648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:11.840648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:11.840648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:11.840648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:11.840648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:11.843026 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:11.843026 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:11.843026 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:11.843026 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:11.845086 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 310/624] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:11.850007 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947f88bc-0785-46ab-a73f-168f93fcd262 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:12.044028 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947f88bc-0785-46ab-a73f-168f93fcd262 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:12.125352 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9bf6d-207f-4bb8-a96d-42d746a9c987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:12.126247 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9bf6d-207f-4bb8-a96d-42d746a9c987 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:12.126701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9bf6d-207f-4bb8-a96d-42d746a9c987 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:12.127140 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9bf6d-207f-4bb8-a96d-42d746a9c987 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:12.128583 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9bf6d-207f-4bb8-a96d-42d746a9c987 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:12.129048 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9bf6d-207f-4bb8-a96d-42d746a9c987 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:12.131520 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9bf6d-207f-4bb8-a96d-42d746a9c987 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:12.153376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 311/625] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:54:12.155386 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77136be1-f137-48a5-b595-61587d10efff None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:12.155540 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fff505-a2bf-453f-8cd3-20c4b8cfbf10 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:12.156081 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fff505-a2bf-453f-8cd3-20c4b8cfbf10 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:12.156387 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fff505-a2bf-453f-8cd3-20c4b8cfbf10 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:12.159124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 312/626] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:12.167734 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fff505-a2bf-453f-8cd3-20c4b8cfbf10 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:12.195523 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:12.210228 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5759239e-be43-4ea5-92bf-b3ac22e81657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:12.210925 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5759239e-be43-4ea5-92bf-b3ac22e81657 None admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:12.211371 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5759239e-be43-4ea5-92bf-b3ac22e81657 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:12.211824 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5759239e-be43-4ea5-92bf-b3ac22e81657 None admin] PATH_INFO: `/v3/projects/36f063d8f65d494a89dbbd72c53417c8/users/0cda7f365ba0405daaf86870960f12d2/roles/d17972f7fe97462b8f9f08aed9bc9cb5` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:12.217500 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:12.245188 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5759239e-be43-4ea5-92bf-b3ac22e81657 None admin] RBAC: Authorizing `identity:create_grant(project_id=36f063d8f65d494a89dbbd72c53417c8, user_id=0cda7f365ba0405daaf86870960f12d2, role_id=d17972f7fe97462b8f9f08aed9bc9cb5)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:12.248893 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5759239e-be43-4ea5-92bf-b3ac22e81657 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:12.275152 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 315/627] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 13:54:11 2022] PUT /identity/v3/projects/36f063d8f65d494a89dbbd72c53417c8/users/0cda7f365ba0405daaf86870960f12d2/roles/d17972f7fe97462b8f9f08aed9bc9cb5 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:54:12.290614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e214a1-d4ee-4c3e-8c89-2a2e7cf670f7 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:12.291671 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e214a1-d4ee-4c3e-8c89-2a2e7cf670f7 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:12.292460 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e214a1-d4ee-4c3e-8c89-2a2e7cf670f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:12.327239 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-a1e214a1-d4ee-4c3e-8c89-2a2e7cf670f7 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:12.330076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a1e214a1-d4ee-4c3e-8c89-2a2e7cf670f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:12.485575 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 316/628] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:54:12.497490 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fff505-a2bf-453f-8cd3-20c4b8cfbf10 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:12.532898 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e214a1-d4ee-4c3e-8c89-2a2e7cf670f7 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:12.608105 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a661a28-7afb-4623-b8f0-e9b3f6d9b2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:12.609922 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a661a28-7afb-4623-b8f0-e9b3f6d9b2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:12.610335 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a661a28-7afb-4623-b8f0-e9b3f6d9b2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:12.610933 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a661a28-7afb-4623-b8f0-e9b3f6d9b2b4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:12.625375 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a661a28-7afb-4623-b8f0-e9b3f6d9b2b4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:12.628010 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a661a28-7afb-4623-b8f0-e9b3f6d9b2b4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:12.651802 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-7a661a28-7afb-4623-b8f0-e9b3f6d9b2b4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:54:12.651802 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:12.651802 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:12.651802 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:12.651802 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:12.651802 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:12.651802 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:12.651802 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:12.651802 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:12.651802 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:12.651802 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:12.651802 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:12.651802 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:54:12.651802 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:54:12.651802 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:54:12.651802 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:54:12.653753 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:12.653753 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:12.653753 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:54:12.653753 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:12.653753 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:54:12.653753 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:12.653753 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:12.653753 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:12.653753 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:12.653753 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:12.653753 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:12.653753 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:12.653753 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:12.653753 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:12.653753 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:12.653753 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:12.653753 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:12.653753 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:12.655758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:12.655758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:12.655758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:54:12.655758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:54:12.655758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:54:12.655758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:54:12.655758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:54:12.655758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:54:12.655758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:54:12.655758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:12.655758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 313/629] 10.222.0.45 () {56 vars in 1093 bytes} [Mon Aug 1 13:54:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 487 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:12.666106 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a661a28-7afb-4623-b8f0-e9b3f6d9b2b4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:12.766374 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e214a1-d4ee-4c3e-8c89-2a2e7cf670f7 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:12.769786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:12.831708 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:12.841871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db757e-1fd6-4eb6-aec5-bf11670b4377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:12.844334 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db757e-1fd6-4eb6-aec5-bf11670b4377 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:12.845128 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db757e-1fd6-4eb6-aec5-bf11670b4377 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:12.845962 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db757e-1fd6-4eb6-aec5-bf11670b4377 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:12.868649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db757e-1fd6-4eb6-aec5-bf11670b4377 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:54:12.873224 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a661a28-7afb-4623-b8f0-e9b3f6d9b2b4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:12.878966 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-79db757e-1fd6-4eb6-aec5-bf11670b4377 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:54:12.878966 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:12.878966 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:12.878966 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:12.878966 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:12.878966 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:12.878966 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:12.878966 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:12.878966 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:12.878966 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:12.878966 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:12.878966 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:12.878966 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:54:12.878966 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:54:12.878966 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:54:12.878966 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:54:12.880786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:54:12.880786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:54:12.880786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:54:12.880786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:54:12.880786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:12.880786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:12.880786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:12.880786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:12.880786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:12.880786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:12.880786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:12.880786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:12.880786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:12.880786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:12.880786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:12.880786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:12.882360 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:12.882360 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:12.882360 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:12.882360 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:12.882360 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:54:12.882360 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:54:12.882360 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:54:12.882360 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:54:12.882360 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:54:12.882360 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:54:12.882360 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:54:12.882360 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:12.886723 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 317/630] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 13:54:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:54:12.897662 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db757e-1fd6-4eb6-aec5-bf11670b4377 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:12.940018 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44110a2-b03c-4720-9d1d-64f17e227b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:12.941261 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44110a2-b03c-4720-9d1d-64f17e227b9a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:12.941718 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44110a2-b03c-4720-9d1d-64f17e227b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:12.942161 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44110a2-b03c-4720-9d1d-64f17e227b9a None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:12.943562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44110a2-b03c-4720-9d1d-64f17e227b9a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:12.944061 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44110a2-b03c-4720-9d1d-64f17e227b9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:12.947100 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44110a2-b03c-4720-9d1d-64f17e227b9a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:12.985610 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 314/631] 10.222.0.45 () {56 vars in 1133 bytes} [Mon Aug 1 13:54:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:12.995363 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44110a2-b03c-4720-9d1d-64f17e227b9a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:13.200902 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44110a2-b03c-4720-9d1d-64f17e227b9a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:13.276445 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244a0b6-6251-45d0-948e-0b3d532fe6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:13.278350 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244a0b6-6251-45d0-948e-0b3d532fe6ac None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:13.279521 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244a0b6-6251-45d0-948e-0b3d532fe6ac None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:13.280315 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244a0b6-6251-45d0-948e-0b3d532fe6ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:13.292084 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:13.298800 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db757e-1fd6-4eb6-aec5-bf11670b4377 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:13.383284 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f164f-1b51-45a5-9519-8834940e3b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:13.385028 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f164f-1b51-45a5-9519-8834940e3b90 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:13.385827 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f164f-1b51-45a5-9519-8834940e3b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:13.386558 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f164f-1b51-45a5-9519-8834940e3b90 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:13.388505 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f164f-1b51-45a5-9519-8834940e3b90 None admin] RBAC: Adding query filter params (name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:13.389359 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f164f-1b51-45a5-9519-8834940e3b90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:13.392408 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f164f-1b51-45a5-9519-8834940e3b90 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:13.410572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 318/632] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 13:54:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 514 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:13.413532 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65925d-fde1-4eb4-804f-15b521d0b0e3 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:13.414437 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65925d-fde1-4eb4-804f-15b521d0b0e3 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:13.415481 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65925d-fde1-4eb4-804f-15b521d0b0e3 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:13.426037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 319/633] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:13.428350 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e65925d-fde1-4eb4-804f-15b521d0b0e3 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:13.444740 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244a0b6-6251-45d0-948e-0b3d532fe6ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:13.448495 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244a0b6-6251-45d0-948e-0b3d532fe6ac None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:13.484104 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:13.651222 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e65925d-fde1-4eb4-804f-15b521d0b0e3 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:13.684595 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 315/634] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:54:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 690 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:54:13.687451 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a042f-e707-4d82-b607-7e4df7b84578 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:13.687950 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a042f-e707-4d82-b607-7e4df7b84578 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:13.688437 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a042f-e707-4d82-b607-7e4df7b84578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:13.722411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-ba0a042f-e707-4d82-b607-7e4df7b84578 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:13.724718 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ba0a042f-e707-4d82-b607-7e4df7b84578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:13.735067 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf0ab0-2e7a-4db1-8d39-8e612889fad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:13.736681 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf0ab0-2e7a-4db1-8d39-8e612889fad0 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:13.737458 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf0ab0-2e7a-4db1-8d39-8e612889fad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:13.738210 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf0ab0-2e7a-4db1-8d39-8e612889fad0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:13.761899 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdf0ab0-2e7a-4db1-8d39-8e612889fad0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:13.765676 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdf0ab0-2e7a-4db1-8d39-8e612889fad0 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:13.801353 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:13.806658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-9bdf0ab0-2e7a-4db1-8d39-8e612889fad0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:13.806658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:13.806658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:13.806658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:13.806658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:13.806658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:13.806658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:13.806658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:13.806658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:13.806658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:13.806658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:13.806658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:13.806658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:13.806658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:13.806658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:13.808744 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:13.808744 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:13.808744 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:13.808744 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:13.808744 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:13.808744 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:13.808744 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:13.808744 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:13.808744 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:13.808744 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:13.808744 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:13.808744 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:13.808744 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:13.808744 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:13.808744 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:13.808744 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:13.808744 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:13.810427 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:13.810427 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:13.810427 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:13.810427 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:13.814043 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 320/635] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:13.815952 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf0ab0-2e7a-4db1-8d39-8e612889fad0 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:13.899752 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 316/636] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:54:13.899752 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a042f-e707-4d82-b607-7e4df7b84578 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:13.992384 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:14.047624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf0ab0-2e7a-4db1-8d39-8e612889fad0 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:14.121948 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a042f-e707-4d82-b607-7e4df7b84578 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:14.137534 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596a75f-cf03-43cd-9561-16cf92d7b736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:14.139771 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596a75f-cf03-43cd-9561-16cf92d7b736 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:14.140522 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596a75f-cf03-43cd-9561-16cf92d7b736 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:14.141112 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596a75f-cf03-43cd-9561-16cf92d7b736 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:14.163296 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d596a75f-cf03-43cd-9561-16cf92d7b736 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:14.166535 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d596a75f-cf03-43cd-9561-16cf92d7b736 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:14.196819 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-d596a75f-cf03-43cd-9561-16cf92d7b736 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:14.196819 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:14.196819 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:14.196819 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:14.196819 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:14.196819 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:14.196819 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:14.196819 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:14.196819 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:14.196819 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:14.196819 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:14.196819 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:14.196819 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:14.196819 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:14.196819 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:14.198552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:14.198552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:14.198552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:14.198552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:14.198552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:14.198552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:14.198552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:14.198552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:14.198552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:14.198552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:14.198552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:14.198552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:14.198552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:14.198552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:14.198552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:14.198552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:14.198552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:14.200321 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:14.200321 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:14.200321 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:14.200321 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:14.200761 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4d271-6f8e-431f-b287-10ff38593d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:14.200761 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4d271-6f8e-431f-b287-10ff38593d65 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:14.200761 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4d271-6f8e-431f-b287-10ff38593d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:14.200761 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4d271-6f8e-431f-b287-10ff38593d65 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:14.202684 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4d271-6f8e-431f-b287-10ff38593d65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:14.203331 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4d271-6f8e-431f-b287-10ff38593d65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:14.205663 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 321/637] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:14.206211 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4d271-6f8e-431f-b287-10ff38593d65 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:14.207777 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596a75f-cf03-43cd-9561-16cf92d7b736 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:14.232932 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 317/638] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:14.235418 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4d271-6f8e-431f-b287-10ff38593d65 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:14.306271 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:14.434571 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596a75f-cf03-43cd-9561-16cf92d7b736 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:14.446613 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4d271-6f8e-431f-b287-10ff38593d65 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:14.492598 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:14.514585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49640a3d-bae9-4c94-b44f-6b6cfcbebc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:14.515768 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49640a3d-bae9-4c94-b44f-6b6cfcbebc44 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:14.516266 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49640a3d-bae9-4c94-b44f-6b6cfcbebc44 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:14.516799 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49640a3d-bae9-4c94-b44f-6b6cfcbebc44 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:14.518787 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9a1c1-7e38-4243-a698-7b0ebf757635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:14.520054 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9a1c1-7e38-4243-a698-7b0ebf757635 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:14.520685 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9a1c1-7e38-4243-a698-7b0ebf757635 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:14.521171 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9a1c1-7e38-4243-a698-7b0ebf757635 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:14.522615 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd9a1c1-7e38-4243-a698-7b0ebf757635 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:14.523157 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd9a1c1-7e38-4243-a698-7b0ebf757635 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:14.525982 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd9a1c1-7e38-4243-a698-7b0ebf757635 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:14.549007 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49640a3d-bae9-4c94-b44f-6b6cfcbebc44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:54:14.565453 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 318/639] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:14.565780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-49640a3d-bae9-4c94-b44f-6b6cfcbebc44 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:54:14.565780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:14.565780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:14.565780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:14.565780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:14.565780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:14.565780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:14.565780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:14.565780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:14.565780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:14.565780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:14.565780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:14.565780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:54:14.565780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:54:14.565780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:54:14.565780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:54:14.567803 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:54:14.567803 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:54:14.567803 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:54:14.567803 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:54:14.567803 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:14.567803 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:14.567803 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:14.567803 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:14.567803 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:14.567803 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:14.567803 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:14.567803 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:14.567803 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:14.567803 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:14.567803 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:14.567803 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:14.570556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:14.570556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:14.570556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:14.570556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:14.570556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:54:14.570556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:54:14.570556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:54:14.570556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:54:14.570556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:54:14.570556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:54:14.570556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:54:14.570556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:14.571783 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9a1c1-7e38-4243-a698-7b0ebf757635 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:14.571935 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 322/640] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 13:54:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:14.571935 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adfe46b-3d15-455e-a04a-2ffddcb4fcee None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:14.571935 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adfe46b-3d15-455e-a04a-2ffddcb4fcee None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:14.571935 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adfe46b-3d15-455e-a04a-2ffddcb4fcee None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:14.571935 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 323/641] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:14.571935 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adfe46b-3d15-455e-a04a-2ffddcb4fcee None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:14.720451 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:14.751600 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9a1c1-7e38-4243-a698-7b0ebf757635 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:14.790862 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adfe46b-3d15-455e-a04a-2ffddcb4fcee None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:14.818330 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03e5cd-1f7d-41d3-b77d-bdb84dd1b9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:14.819303 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03e5cd-1f7d-41d3-b77d-bdb84dd1b9a7 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:14.819796 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03e5cd-1f7d-41d3-b77d-bdb84dd1b9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:14.820343 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03e5cd-1f7d-41d3-b77d-bdb84dd1b9a7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:14.834350 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c03e5cd-1f7d-41d3-b77d-bdb84dd1b9a7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:14.837109 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c03e5cd-1f7d-41d3-b77d-bdb84dd1b9a7 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:14.854030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-0c03e5cd-1f7d-41d3-b77d-bdb84dd1b9a7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 13:54:14.854030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:14.854030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:14.854030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:14.854030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:14.854030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:14.854030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:14.854030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:14.854030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:14.854030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:14.854030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:14.854030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:14.854030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:54:14.854030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:54:14.854030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:54:14.854030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:54:14.855240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:14.855240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:14.855240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:54:14.855240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:14.855240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:54:14.855240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:14.855240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:14.855240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:14.855240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:14.855240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:14.855240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:14.855240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:14.855240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:14.855240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:14.855240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:14.855240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:14.855240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:14.855240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:14.856527 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:14.856527 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:14.856527 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:54:14.856527 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:54:14.856527 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:54:14.856527 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:54:14.856527 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:54:14.856527 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:54:14.856527 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 13:54:14.856527 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:14.863887 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 319/642] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:54:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:14.864214 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe5728-b506-427d-817d-169b637550e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:14.864471 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03e5cd-1f7d-41d3-b77d-bdb84dd1b9a7 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:14.864627 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe5728-b506-427d-817d-169b637550e5 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:14.864627 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe5728-b506-427d-817d-169b637550e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:14.864627 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe5728-b506-427d-817d-169b637550e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:14.875250 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbe5728-b506-427d-817d-169b637550e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:14.877868 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbe5728-b506-427d-817d-169b637550e5 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:14.900863 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-7dbe5728-b506-427d-817d-169b637550e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:14.900863 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:14.900863 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:14.900863 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:14.900863 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:14.900863 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:14.900863 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:14.900863 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:14.900863 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:14.900863 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:14.900863 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:14.900863 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:14.900863 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:54:14.900863 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:54:14.900863 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:54:14.901871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:54:14.901871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:14.901871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:14.901871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:14.901871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:14.901871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:14.901871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:14.901871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:14.901871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:14.901871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:14.901871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:14.901871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:14.901871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:14.901871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:14.901871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:14.901871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:14.901871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:14.902833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:54:14.902833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:54:14.902833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:54:14.902833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:54:14.902833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:54:14.902833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:54:14.902833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:14.902833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:14.903800 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 324/643] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:54:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:14.905878 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107542a6-acd3-4195-a3e6-f3f63c8a3b47 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:14.906392 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107542a6-acd3-4195-a3e6-f3f63c8a3b47 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:14.906883 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107542a6-acd3-4195-a3e6-f3f63c8a3b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:14.937587 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-107542a6-acd3-4195-a3e6-f3f63c8a3b47 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:14.939698 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-107542a6-acd3-4195-a3e6-f3f63c8a3b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:14.999519 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:15.029685 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03e5cd-1f7d-41d3-b77d-bdb84dd1b9a7 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:15.047782 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 325/644] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:54:15.050357 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107542a6-acd3-4195-a3e6-f3f63c8a3b47 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:15.081253 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff47d8a3-7204-443f-8a4d-441341e55da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:15.082195 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47d8a3-7204-443f-8a4d-441341e55da8 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:15.082465 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47d8a3-7204-443f-8a4d-441341e55da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:15.082742 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47d8a3-7204-443f-8a4d-441341e55da8 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:15.083474 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff47d8a3-7204-443f-8a4d-441341e55da8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:15.083778 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff47d8a3-7204-443f-8a4d-441341e55da8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:15.085467 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff47d8a3-7204-443f-8a4d-441341e55da8 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:15.097639 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 320/645] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 13:54:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:15.098531 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff47d8a3-7204-443f-8a4d-441341e55da8 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:15.112366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:15.225881 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107542a6-acd3-4195-a3e6-f3f63c8a3b47 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:15.261981 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff47d8a3-7204-443f-8a4d-441341e55da8 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:15.273497 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0ea17-0441-48e7-ae64-445b0fef055c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:15.274367 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0ea17-0441-48e7-ae64-445b0fef055c None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:15.274766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0ea17-0441-48e7-ae64-445b0fef055c None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:15.275155 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0ea17-0441-48e7-ae64-445b0fef055c None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:15.276606 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0ea17-0441-48e7-ae64-445b0fef055c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:15.277086 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0ea17-0441-48e7-ae64-445b0fef055c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:15.279050 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0ea17-0441-48e7-ae64-445b0fef055c None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:15.320198 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 326/646] 10.222.0.45 () {56 vars in 1137 bytes} [Mon Aug 1 13:54:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:15.320198 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0ea17-0441-48e7-ae64-445b0fef055c None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:15.337624 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d12fc-140d-4baf-80d3-767eb1180ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:15.338508 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d12fc-140d-4baf-80d3-767eb1180ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:15.338974 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d12fc-140d-4baf-80d3-767eb1180ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:15.339419 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d12fc-140d-4baf-80d3-767eb1180ed6 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:15.340746 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d12fc-140d-4baf-80d3-767eb1180ed6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:15.341237 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d12fc-140d-4baf-80d3-767eb1180ed6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:15.344075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d12fc-140d-4baf-80d3-767eb1180ed6 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:15.366542 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 321/647] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:54:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:15.373824 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d12fc-140d-4baf-80d3-767eb1180ed6 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:15.388519 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:15.511375 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0ea17-0441-48e7-ae64-445b0fef055c None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:15.516584 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:15.568534 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d12fc-140d-4baf-80d3-767eb1180ed6 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:15.575508 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b107d716-df29-4217-9fe7-c16a4e5fb18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:15.576755 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107d716-df29-4217-9fe7-c16a4e5fb18e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:15.577314 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107d716-df29-4217-9fe7-c16a4e5fb18e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:15.578512 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107d716-df29-4217-9fe7-c16a4e5fb18e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:15.598920 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b107d716-df29-4217-9fe7-c16a4e5fb18e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:54:15.608438 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-b107d716-df29-4217-9fe7-c16a4e5fb18e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:54:15.608438 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:15.608438 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:15.608438 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:15.608438 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:15.608438 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:15.608438 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:15.608438 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:15.608438 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:15.608438 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:15.608438 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:15.608438 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:15.608438 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:54:15.608438 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:54:15.608438 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:54:15.608438 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:54:15.610376 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:54:15.610376 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:54:15.610376 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:54:15.610376 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:54:15.610376 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:15.610376 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:15.610376 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:15.610376 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:15.610376 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:15.610376 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:15.610376 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:15.610376 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:15.610376 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:15.610376 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:15.610376 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:15.610376 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:15.612011 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:15.612011 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:15.612011 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:15.612011 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:15.612011 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:54:15.612011 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:54:15.612011 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:54:15.612011 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:54:15.612011 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:54:15.612011 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:54:15.612011 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:54:15.612011 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:15.612011 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 327/648] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 13:54:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:15.613421 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b107d716-df29-4217-9fe7-c16a4e5fb18e None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:15.617613 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de90cd14-f03d-4ef5-ba14-afc8d8d6e2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:15.618290 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90cd14-f03d-4ef5-ba14-afc8d8d6e2e7 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:15.618570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90cd14-f03d-4ef5-ba14-afc8d8d6e2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:15.618829 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90cd14-f03d-4ef5-ba14-afc8d8d6e2e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:15.632828 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de90cd14-f03d-4ef5-ba14-afc8d8d6e2e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:15.634340 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de90cd14-f03d-4ef5-ba14-afc8d8d6e2e7 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:15.653053 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-de90cd14-f03d-4ef5-ba14-afc8d8d6e2e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:15.653053 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:15.653053 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:15.653053 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:15.653053 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:15.653053 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:15.653053 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:15.653053 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:15.653053 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:15.653053 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:15.653053 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:15.653053 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:15.653053 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:15.653053 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:15.653053 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:15.654803 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:15.654803 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:15.654803 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:15.654803 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:15.654803 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:15.654803 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:15.654803 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:15.654803 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:15.654803 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:15.654803 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:15.654803 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:15.654803 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:15.654803 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:15.654803 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:15.654803 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:15.654803 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:15.654803 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:15.657002 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:15.657002 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:15.657002 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:15.657002 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:15.657002 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 322/649] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:15.657002 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de90cd14-f03d-4ef5-ba14-afc8d8d6e2e7 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:15.786769 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:15.795508 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b107d716-df29-4217-9fe7-c16a4e5fb18e None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:15.857878 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b98f1-892d-45aa-95c1-627dcaaf85ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:15.859528 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b98f1-892d-45aa-95c1-627dcaaf85ab None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:15.859707 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de90cd14-f03d-4ef5-ba14-afc8d8d6e2e7 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:15.860467 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b98f1-892d-45aa-95c1-627dcaaf85ab None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:15.861156 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b98f1-892d-45aa-95c1-627dcaaf85ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:15.902093 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:15.929660 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb4b508-eeb0-45d3-8511-92cc24f61d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:15.930807 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4b508-eeb0-45d3-8511-92cc24f61d3f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:15.931283 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4b508-eeb0-45d3-8511-92cc24f61d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:15.931753 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4b508-eeb0-45d3-8511-92cc24f61d3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:15.933256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb4b508-eeb0-45d3-8511-92cc24f61d3f None admin] RBAC: Adding query filter params (role.id=d17972f7fe97462b8f9f08aed9bc9cb5, scope.project.id=36f063d8f65d494a89dbbd72c53417c8, user.id=5b3adc14dcaf4ac79b43bb97fa1ba1d4) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:15.933841 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb4b508-eeb0-45d3-8511-92cc24f61d3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:15.936612 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb4b508-eeb0-45d3-8511-92cc24f61d3f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:15.954107 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 323/650] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:54:15 2022] GET /identity/v3/role_assignments?user.id=5b3adc14dcaf4ac79b43bb97fa1ba1d4&scope.project.id=36f063d8f65d494a89dbbd72c53417c8&role.id=d17972f7fe97462b8f9f08aed9bc9cb5 => generated 608 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:15.954107 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb4b508-eeb0-45d3-8511-92cc24f61d3f None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:16.011572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b98f1-892d-45aa-95c1-627dcaaf85ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:16.015438 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b98f1-892d-45aa-95c1-627dcaaf85ab None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:16.154450 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb4b508-eeb0-45d3-8511-92cc24f61d3f None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:16.170195 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:16.189410 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 328/651] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:54:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:54:16.189410 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b98f1-892d-45aa-95c1-627dcaaf85ab None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:16.238045 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59323dbe-cdac-4704-b8bb-f027a21bed6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:16.239199 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59323dbe-cdac-4704-b8bb-f027a21bed6f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:16.239738 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59323dbe-cdac-4704-b8bb-f027a21bed6f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:16.240367 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59323dbe-cdac-4704-b8bb-f027a21bed6f None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:16.241548 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59323dbe-cdac-4704-b8bb-f027a21bed6f None admin] RBAC: Adding query filter params (name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:16.242057 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59323dbe-cdac-4704-b8bb-f027a21bed6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:16.244643 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59323dbe-cdac-4704-b8bb-f027a21bed6f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:16.259714 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 324/652] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 13:54:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:16.261569 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b80d3c-bd25-45a9-abc6-120b76c8450d None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:16.262157 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b80d3c-bd25-45a9-abc6-120b76c8450d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:16.262705 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b80d3c-bd25-45a9-abc6-120b76c8450d None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:16.264930 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 325/653] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:16.266227 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b80d3c-bd25-45a9-abc6-120b76c8450d None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:16.371322 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:16.377508 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b98f1-892d-45aa-95c1-627dcaaf85ab None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:16.447580 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d349ebc-3f98-43f3-bc5d-6b6921c900f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:16.448642 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d349ebc-3f98-43f3-bc5d-6b6921c900f2 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:16.449110 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d349ebc-3f98-43f3-bc5d-6b6921c900f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:16.449586 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d349ebc-3f98-43f3-bc5d-6b6921c900f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:16.450917 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d349ebc-3f98-43f3-bc5d-6b6921c900f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:16.451461 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d349ebc-3f98-43f3-bc5d-6b6921c900f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:16.454140 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d349ebc-3f98-43f3-bc5d-6b6921c900f2 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:16.482856 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 329/654] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:16.482856 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d349ebc-3f98-43f3-bc5d-6b6921c900f2 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:16.498524 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b80d3c-bd25-45a9-abc6-120b76c8450d None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:16.568332 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209bc137-ce33-4447-9fae-9e8a5669631e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:16.568573 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:16.569685 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209bc137-ce33-4447-9fae-9e8a5669631e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:16.570250 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209bc137-ce33-4447-9fae-9e8a5669631e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:16.570611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209bc137-ce33-4447-9fae-9e8a5669631e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:16.589227 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209bc137-ce33-4447-9fae-9e8a5669631e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:16.591690 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209bc137-ce33-4447-9fae-9e8a5669631e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:16.616547 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-209bc137-ce33-4447-9fae-9e8a5669631e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:16.616547 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:16.616547 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:16.616547 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:16.616547 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:16.616547 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:16.616547 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:16.616547 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:16.616547 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:16.616547 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:16.616547 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:16.616547 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:16.616547 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:16.616547 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:16.616547 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:16.618570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:16.618570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:16.618570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:16.618570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:16.618570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:16.618570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:16.618570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:16.618570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:16.618570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:16.618570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:16.618570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:16.618570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:16.618570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:16.618570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:16.618570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:16.618570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:16.618570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:16.622121 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:16.622121 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:16.622121 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:16.622121 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:16.622121 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 326/655] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:16.622121 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d00460-7cfe-4e73-9406-40e5f68d3676 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:16.622121 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d00460-7cfe-4e73-9406-40e5f68d3676 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:16.622121 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d00460-7cfe-4e73-9406-40e5f68d3676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:16.645806 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-e5d00460-7cfe-4e73-9406-40e5f68d3676 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:16.647225 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e5d00460-7cfe-4e73-9406-40e5f68d3676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:16.659093 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d349ebc-3f98-43f3-bc5d-6b6921c900f2 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:16.718624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ecb1df-af33-45b4-b2c3-e031fb64a197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:16.719868 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ecb1df-af33-45b4-b2c3-e031fb64a197 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:16.720246 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ecb1df-af33-45b4-b2c3-e031fb64a197 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:16.720246 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ecb1df-af33-45b4-b2c3-e031fb64a197 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:16.744237 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ecb1df-af33-45b4-b2c3-e031fb64a197 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:16.747871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ecb1df-af33-45b4-b2c3-e031fb64a197 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:16.777193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-c2ecb1df-af33-45b4-b2c3-e031fb64a197 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:16.777193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:16.777193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:16.777193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:16.777193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:16.777193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:16.777193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:16.777193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:16.777193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:16.777193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:16.777193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:16.777193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:16.777193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:16.777193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:16.777193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:16.779101 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:16.779101 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:16.779101 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:16.779101 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:16.779101 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:16.779101 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:16.779101 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:16.779101 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:16.779101 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:16.779101 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:16.779101 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:16.779101 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:16.779101 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:16.779101 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:16.779101 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:16.779101 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:16.779101 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:16.781032 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:16.781032 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:16.781032 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:16.781032 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:16.781032 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 330/656] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:16.781032 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ecb1df-af33-45b4-b2c3-e031fb64a197 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:16.790464 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 327/657] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:54:16.792355 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d00460-7cfe-4e73-9406-40e5f68d3676 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:16.844974 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:16.979983 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ecb1df-af33-45b4-b2c3-e031fb64a197 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:16.987289 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d00460-7cfe-4e73-9406-40e5f68d3676 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:16.999352 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:17.047784 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3793d-14fe-4cda-9512-7444daeed323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:17.048998 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3793d-14fe-4cda-9512-7444daeed323 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:17.049461 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3793d-14fe-4cda-9512-7444daeed323 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:17.049940 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3793d-14fe-4cda-9512-7444daeed323 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:17.057736 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0beb56-8cea-4283-b93f-4a89e8ab6a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:17.058736 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0beb56-8cea-4283-b93f-4a89e8ab6a2b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:17.059224 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0beb56-8cea-4283-b93f-4a89e8ab6a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:17.059674 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0beb56-8cea-4283-b93f-4a89e8ab6a2b None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:17.061111 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0beb56-8cea-4283-b93f-4a89e8ab6a2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:17.061605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0beb56-8cea-4283-b93f-4a89e8ab6a2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:17.064081 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0beb56-8cea-4283-b93f-4a89e8ab6a2b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:17.065832 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb3793d-14fe-4cda-9512-7444daeed323 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:17.068637 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb3793d-14fe-4cda-9512-7444daeed323 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:17.086212 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 328/658] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:17.088072 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0beb56-8cea-4283-b93f-4a89e8ab6a2b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:17.091875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-ceb3793d-14fe-4cda-9512-7444daeed323 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 13:54:17.091875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:17.091875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:17.091875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:17.091875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:17.091875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:17.091875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:17.091875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:17.091875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:17.091875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:17.091875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:17.091875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:17.091875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:54:17.091875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:54:17.091875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:54:17.091875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:54:17.093731 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:17.093731 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:17.093731 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:54:17.093731 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:17.093731 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:54:17.093731 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:17.093731 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:17.093731 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:17.093731 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:17.093731 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:17.093731 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:17.093731 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:17.093731 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:17.093731 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:17.093731 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:17.093731 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:17.093731 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:17.093731 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:17.095411 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:17.095411 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:17.095411 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:54:17.095411 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:54:17.095411 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:54:17.095411 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:54:17.095411 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:54:17.095411 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:54:17.095411 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 13:54:17.095411 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:17.100888 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 331/659] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:54:16 2022] GET /identity/v3/users/glance => generated 84 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:17.102891 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3793d-14fe-4cda-9512-7444daeed323 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:17.272792 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:17.298067 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0beb56-8cea-4283-b93f-4a89e8ab6a2b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:17.316386 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3793d-14fe-4cda-9512-7444daeed323 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:17.373810 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2078d7a4-f995-4843-b224-d04602763d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:17.374897 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078d7a4-f995-4843-b224-d04602763d3c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:17.375388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078d7a4-f995-4843-b224-d04602763d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:17.375853 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078d7a4-f995-4843-b224-d04602763d3c None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:17.377199 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2078d7a4-f995-4843-b224-d04602763d3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:17.377750 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2078d7a4-f995-4843-b224-d04602763d3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:17.380328 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2078d7a4-f995-4843-b224-d04602763d3c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:17.386167 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf38089-b0b2-4c9b-b494-25b5842bf24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:17.393214 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf38089-b0b2-4c9b-b494-25b5842bf24b None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:17.393214 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf38089-b0b2-4c9b-b494-25b5842bf24b None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:17.393214 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf38089-b0b2-4c9b-b494-25b5842bf24b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:17.410783 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 329/660] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:17.410783 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2078d7a4-f995-4843-b224-d04602763d3c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:17.421461 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf38089-b0b2-4c9b-b494-25b5842bf24b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:54:17.432177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-bcf38089-b0b2-4c9b-b494-25b5842bf24b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:54:17.432177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:17.432177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:17.432177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:17.432177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:17.432177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:17.432177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:17.432177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:17.432177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:17.432177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:17.432177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:17.432177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:17.432177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:54:17.432177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:54:17.432177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:54:17.432177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:54:17.434057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:54:17.434057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:54:17.434057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:54:17.434057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:54:17.434057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:17.434057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:17.434057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:17.434057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:17.434057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:17.434057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:17.434057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:17.434057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:17.434057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:17.434057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:17.434057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:17.434057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:17.435608 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:17.435608 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:17.435608 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:17.435608 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:17.435608 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:54:17.435608 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:54:17.435608 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:54:17.435608 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:54:17.435608 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:54:17.435608 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:54:17.435608 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:54:17.435608 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:17.435608 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 332/661] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 13:54:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:17.438303 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf38089-b0b2-4c9b-b494-25b5842bf24b None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:17.490593 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:17.607630 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2078d7a4-f995-4843-b224-d04602763d3c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:17.629907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf38089-b0b2-4c9b-b494-25b5842bf24b None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:17.670534 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea262fbf-8c84-48b6-adfc-3b5fe3df118c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:17.671723 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea262fbf-8c84-48b6-adfc-3b5fe3df118c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:17.672133 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea262fbf-8c84-48b6-adfc-3b5fe3df118c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:17.672605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea262fbf-8c84-48b6-adfc-3b5fe3df118c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:17.681186 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea262fbf-8c84-48b6-adfc-3b5fe3df118c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:17.683663 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea262fbf-8c84-48b6-adfc-3b5fe3df118c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:17.684931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b3bec-a137-4b20-8672-e80adbb13fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:17.685315 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b3bec-a137-4b20-8672-e80adbb13fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:17.685754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b3bec-a137-4b20-8672-e80adbb13fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:17.686181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b3bec-a137-4b20-8672-e80adbb13fc0 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:17.687710 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62b3bec-a137-4b20-8672-e80adbb13fc0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:17.688095 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62b3bec-a137-4b20-8672-e80adbb13fc0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:17.690579 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62b3bec-a137-4b20-8672-e80adbb13fc0 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:17.704748 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-ea262fbf-8c84-48b6-adfc-3b5fe3df118c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:17.704748 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:17.704748 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:17.704748 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:17.704748 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:17.704748 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:17.704748 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:17.704748 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:17.704748 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:17.704748 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:17.704748 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:17.704748 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:17.704748 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:54:17.704748 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:54:17.704748 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:54:17.706597 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:54:17.706597 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:17.706597 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:17.706597 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:17.706597 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:17.706597 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:17.706597 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:17.706597 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:17.706597 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:17.706597 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:17.706597 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:17.706597 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:17.706597 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:17.706597 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:17.706597 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:17.706597 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:17.706597 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:17.709479 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:54:17.709479 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:54:17.709479 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:54:17.709479 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:54:17.709479 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:54:17.709479 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:54:17.709479 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:17.709479 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:17.709479 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 330/662] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:54:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:17.709479 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea262fbf-8c84-48b6-adfc-3b5fe3df118c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:17.725458 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:17.743680 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 333/663] 10.222.0.45 () {56 vars in 1137 bytes} [Mon Aug 1 13:54:17 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:17.743680 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b3bec-a137-4b20-8672-e80adbb13fc0 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:17.894723 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea262fbf-8c84-48b6-adfc-3b5fe3df118c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:17.922967 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:17.955011 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b3bec-a137-4b20-8672-e80adbb13fc0 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:17.971376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc70d5-2436-437d-a185-09350db6ffdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:17.972402 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc70d5-2436-437d-a185-09350db6ffdf None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:17.972870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc70d5-2436-437d-a185-09350db6ffdf None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:17.973328 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc70d5-2436-437d-a185-09350db6ffdf None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:17.986325 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc70d5-2436-437d-a185-09350db6ffdf None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:17.988863 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc70d5-2436-437d-a185-09350db6ffdf None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:18.009537 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-6adc70d5-2436-437d-a185-09350db6ffdf None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 13:54:18.009537 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:18.009537 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:18.009537 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:18.009537 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:18.009537 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:18.009537 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:18.009537 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:18.009537 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:18.009537 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:18.009537 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:18.009537 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:18.009537 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:54:18.009537 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:54:18.009537 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:54:18.009537 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:54:18.011379 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:18.011379 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:18.011379 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:54:18.011379 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:18.011379 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:54:18.011379 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:18.011379 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:18.011379 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:18.011379 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:18.011379 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:18.011379 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:18.011379 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:18.011379 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:18.011379 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:18.011379 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:18.011379 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:18.011379 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:18.011379 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:18.013246 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:18.013246 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:18.013246 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:54:18.013246 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:54:18.013246 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:54:18.013246 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:54:18.013246 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:54:18.013246 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:54:18.013246 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 13:54:18.013246 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:18.013246 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 331/664] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 13:54:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:18.013246 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc70d5-2436-437d-a185-09350db6ffdf None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:18.025304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3a135-a32f-4fbd-83ca-68d9efac7d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:18.026285 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3a135-a32f-4fbd-83ca-68d9efac7d63 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:18.026792 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3a135-a32f-4fbd-83ca-68d9efac7d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:18.027224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3a135-a32f-4fbd-83ca-68d9efac7d63 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:18.028485 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce3a135-a32f-4fbd-83ca-68d9efac7d63 None admin] RBAC: Adding query filter params (name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:18.028961 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce3a135-a32f-4fbd-83ca-68d9efac7d63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:18.031408 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce3a135-a32f-4fbd-83ca-68d9efac7d63 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:18.046048 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 334/665] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 13:54:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:18.053718 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3a135-a32f-4fbd-83ca-68d9efac7d63 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:18.116826 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:18.179133 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc70d5-2436-437d-a185-09350db6ffdf None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:18.241912 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32937fb6-cd2f-42e2-95f3-a635dcaf33eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:18.242970 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32937fb6-cd2f-42e2-95f3-a635dcaf33eb None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:18.243458 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32937fb6-cd2f-42e2-95f3-a635dcaf33eb None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:18.243933 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32937fb6-cd2f-42e2-95f3-a635dcaf33eb None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:18.245368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32937fb6-cd2f-42e2-95f3-a635dcaf33eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:18.245922 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32937fb6-cd2f-42e2-95f3-a635dcaf33eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:18.246034 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3a135-a32f-4fbd-83ca-68d9efac7d63 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:18.249019 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32937fb6-cd2f-42e2-95f3-a635dcaf33eb None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:18.274075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 332/666] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:54:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:18.275585 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32937fb6-cd2f-42e2-95f3-a635dcaf33eb None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:18.326980 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0835fb-2336-4430-9139-a2ebb96f6ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:18.327937 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0835fb-2336-4430-9139-a2ebb96f6ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:18.328450 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0835fb-2336-4430-9139-a2ebb96f6ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:18.328916 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0835fb-2336-4430-9139-a2ebb96f6ae2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:18.365156 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:18.452174 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0835fb-2336-4430-9139-a2ebb96f6ae2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:18.454680 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0835fb-2336-4430-9139-a2ebb96f6ae2 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:18.488352 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32937fb6-cd2f-42e2-95f3-a635dcaf33eb None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:18.548993 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:18.564881 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b4098-5606-4be5-a5e6-54e8d8c10fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:18.565866 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b4098-5606-4be5-a5e6-54e8d8c10fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:18.566358 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b4098-5606-4be5-a5e6-54e8d8c10fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:18.566838 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b4098-5606-4be5-a5e6-54e8d8c10fe0 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:18.568383 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658b4098-5606-4be5-a5e6-54e8d8c10fe0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:18.568938 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658b4098-5606-4be5-a5e6-54e8d8c10fe0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:18.571702 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658b4098-5606-4be5-a5e6-54e8d8c10fe0 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:18.602714 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 333/667] 10.222.0.45 () {56 vars in 1139 bytes} [Mon Aug 1 13:54:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:18.603972 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b4098-5606-4be5-a5e6-54e8d8c10fe0 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:18.675232 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 335/668] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:54:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 622 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:54:18.676734 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0835fb-2336-4430-9139-a2ebb96f6ae2 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:18.784031 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:18.833345 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b4098-5606-4be5-a5e6-54e8d8c10fe0 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:18.879065 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0835fb-2336-4430-9139-a2ebb96f6ae2 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:18.891330 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62288f6b-3e31-47ec-8f7c-9590a4a87ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:18.891998 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62288f6b-3e31-47ec-8f7c-9590a4a87ca4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:18.892316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62288f6b-3e31-47ec-8f7c-9590a4a87ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:18.892586 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62288f6b-3e31-47ec-8f7c-9590a4a87ca4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:18.908065 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62288f6b-3e31-47ec-8f7c-9590a4a87ca4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:18.910677 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62288f6b-3e31-47ec-8f7c-9590a4a87ca4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:18.932939 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-62288f6b-3e31-47ec-8f7c-9590a4a87ca4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:18.932939 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:18.932939 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:18.932939 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:18.932939 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:18.932939 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:18.932939 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:18.932939 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:18.932939 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:18.932939 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:18.932939 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:18.932939 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:18.932939 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:18.932939 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:18.932939 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:18.934288 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:18.934288 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:18.934288 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:18.934288 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:18.934288 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:18.934288 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:18.934288 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:18.934288 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:18.934288 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:18.934288 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:18.934288 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:18.934288 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:18.934288 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:18.934288 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:18.934288 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:18.934288 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:18.934288 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:18.935542 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:18.935542 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:18.935542 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:18.935542 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:18.935542 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 334/669] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:18.935542 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62288f6b-3e31-47ec-8f7c-9590a4a87ca4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:18.952337 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bccbf1c-59a3-40a2-9007-470c7e7e9f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:18.953782 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bccbf1c-59a3-40a2-9007-470c7e7e9f9c None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:18.954077 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bccbf1c-59a3-40a2-9007-470c7e7e9f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:18.954451 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bccbf1c-59a3-40a2-9007-470c7e7e9f9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:18.955789 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bccbf1c-59a3-40a2-9007-470c7e7e9f9c None admin] RBAC: Adding query filter params (role.id=d17972f7fe97462b8f9f08aed9bc9cb5, scope.project.id=36f063d8f65d494a89dbbd72c53417c8, user.id=8f4c3d46ac7243aeb722a4410f40d846) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:18.956398 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bccbf1c-59a3-40a2-9007-470c7e7e9f9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:18.958246 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bccbf1c-59a3-40a2-9007-470c7e7e9f9c None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:18.968407 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 336/670] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:54:18 2022] GET /identity/v3/role_assignments?user.id=8f4c3d46ac7243aeb722a4410f40d846&scope.project.id=36f063d8f65d494a89dbbd72c53417c8&role.id=d17972f7fe97462b8f9f08aed9bc9cb5 => generated 608 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:18.969555 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bccbf1c-59a3-40a2-9007-470c7e7e9f9c None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:19.005552 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:19.098703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62288f6b-3e31-47ec-8f7c-9590a4a87ca4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:19.145987 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3fedd9-bb69-41ae-89f9-54b041175d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:19.146638 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3fedd9-bb69-41ae-89f9-54b041175d23 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:19.146913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3fedd9-bb69-41ae-89f9-54b041175d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:19.147186 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3fedd9-bb69-41ae-89f9-54b041175d23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:19.175118 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bccbf1c-59a3-40a2-9007-470c7e7e9f9c None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:19.204745 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:19.250109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d9decf-ead6-401b-a104-ec536f7f5c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:19.251268 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9decf-ead6-401b-a104-ec536f7f5c18 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:19.251739 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9decf-ead6-401b-a104-ec536f7f5c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:19.252207 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9decf-ead6-401b-a104-ec536f7f5c18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:19.275434 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d9decf-ead6-401b-a104-ec536f7f5c18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:19.277789 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3fedd9-bb69-41ae-89f9-54b041175d23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:19.278593 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d9decf-ead6-401b-a104-ec536f7f5c18 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:19.282175 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3fedd9-bb69-41ae-89f9-54b041175d23 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:19.314725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-06d9decf-ead6-401b-a104-ec536f7f5c18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:19.314725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:19.314725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:19.314725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:19.314725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:19.314725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:19.314725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:19.314725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:19.314725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:19.314725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:19.314725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:19.314725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:19.314725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:19.314725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:19.314725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:19.316334 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:19.316334 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:19.316334 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:19.316334 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:19.316334 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:19.316334 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:19.316334 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:19.316334 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:19.316334 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:19.316334 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:19.316334 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:19.316334 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:19.316334 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:19.316334 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:19.316334 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:19.316334 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:19.316334 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:19.317921 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:19.317921 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:19.317921 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:19.317921 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:19.317921 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 337/671] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:19.319936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d9decf-ead6-401b-a104-ec536f7f5c18 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:19.360216 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:19.458748 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 335/672] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:54:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:54:19.461014 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80266cbd-5226-4cc0-96e6-05cddb9dc0e2 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:19.461573 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80266cbd-5226-4cc0-96e6-05cddb9dc0e2 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:19.462057 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80266cbd-5226-4cc0-96e6-05cddb9dc0e2 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:19.470648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 336/673] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:19 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:19.472234 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80266cbd-5226-4cc0-96e6-05cddb9dc0e2 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:19.497605 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d9decf-ead6-401b-a104-ec536f7f5c18 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:19.565333 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0770ce-1f32-4c73-9a8d-6a3d379299d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:19.566304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0770ce-1f32-4c73-9a8d-6a3d379299d6 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:19.566658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0770ce-1f32-4c73-9a8d-6a3d379299d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:19.567035 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0770ce-1f32-4c73-9a8d-6a3d379299d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:19.567811 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0770ce-1f32-4c73-9a8d-6a3d379299d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:19.568178 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0770ce-1f32-4c73-9a8d-6a3d379299d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:19.569630 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0770ce-1f32-4c73-9a8d-6a3d379299d6 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:19.592346 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 338/674] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:19.593442 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0770ce-1f32-4c73-9a8d-6a3d379299d6 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:19.636560 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:19.650012 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80266cbd-5226-4cc0-96e6-05cddb9dc0e2 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:19.718576 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e3fb6-37ad-4e5e-a348-c96feb65094d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:19.719322 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e3fb6-37ad-4e5e-a348-c96feb65094d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:19.719599 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e3fb6-37ad-4e5e-a348-c96feb65094d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:19.719897 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e3fb6-37ad-4e5e-a348-c96feb65094d None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:19.720656 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e3fb6-37ad-4e5e-a348-c96feb65094d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:19.721038 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e3fb6-37ad-4e5e-a348-c96feb65094d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:19.722420 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e3fb6-37ad-4e5e-a348-c96feb65094d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:19.734108 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:19.744261 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 337/675] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:19.745571 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa2a1cc-125f-4df5-bdb0-cf3f00d38634 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:19.746538 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa2a1cc-125f-4df5-bdb0-cf3f00d38634 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:19.746882 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa2a1cc-125f-4df5-bdb0-cf3f00d38634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:19.761786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0770ce-1f32-4c73-9a8d-6a3d379299d6 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:19.767222 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-9fa2a1cc-125f-4df5-bdb0-cf3f00d38634 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:19.769010 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9fa2a1cc-125f-4df5-bdb0-cf3f00d38634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:19.814109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699e088-2bd2-40ca-b11a-cc6e38c5888e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:19.815227 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699e088-2bd2-40ca-b11a-cc6e38c5888e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:19.815714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699e088-2bd2-40ca-b11a-cc6e38c5888e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:19.816192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699e088-2bd2-40ca-b11a-cc6e38c5888e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:19.837809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699e088-2bd2-40ca-b11a-cc6e38c5888e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:19.840445 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699e088-2bd2-40ca-b11a-cc6e38c5888e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:19.869290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-f699e088-2bd2-40ca-b11a-cc6e38c5888e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:19.869290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:19.869290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:19.869290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:19.869290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:19.869290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:19.869290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:19.869290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:19.869290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:19.869290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:19.869290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:19.869290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:19.869290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:19.869290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:19.869290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:19.870789 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:19.870789 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:19.870789 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:19.870789 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:19.870789 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:19.870789 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:19.870789 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:19.870789 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:19.870789 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:19.870789 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:19.870789 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:19.870789 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:19.870789 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:19.870789 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:19.870789 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:19.870789 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:19.870789 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:19.872236 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:19.872236 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:19.872236 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:19.872236 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:19.872236 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 339/676] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:19.873522 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699e088-2bd2-40ca-b11a-cc6e38c5888e None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:19.900513 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 338/677] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:54:19.902221 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa2a1cc-125f-4df5-bdb0-cf3f00d38634 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:20.036532 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:20.071453 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699e088-2bd2-40ca-b11a-cc6e38c5888e None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:20.126002 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c382db5f-f357-4726-9903-975ae919e362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:20.126863 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382db5f-f357-4726-9903-975ae919e362 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:20.127271 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382db5f-f357-4726-9903-975ae919e362 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:20.127843 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382db5f-f357-4726-9903-975ae919e362 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:20.140512 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c382db5f-f357-4726-9903-975ae919e362 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:20.142132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c382db5f-f357-4726-9903-975ae919e362 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:20.159837 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-c382db5f-f357-4726-9903-975ae919e362 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 13:54:20.159837 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:20.159837 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:20.159837 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:20.159837 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:20.159837 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:20.159837 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:20.159837 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:20.159837 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:20.159837 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:20.159837 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:20.159837 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:20.159837 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:54:20.159837 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:54:20.159837 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:54:20.159837 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:54:20.161402 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:20.161402 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:20.161402 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:54:20.161402 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:20.161402 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:54:20.161402 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:20.161402 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:20.161402 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:20.161402 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:20.161402 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:20.161402 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:20.161402 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:20.161402 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:20.161402 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:20.161402 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:20.161402 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:20.161402 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:20.161402 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:20.163195 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:20.163195 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:20.163195 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:54:20.163195 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:54:20.163195 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:54:20.163195 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:54:20.163195 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:54:20.163195 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:54:20.163195 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 13:54:20.163195 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:20.163195 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 340/678] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:54:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:20.169195 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c382db5f-f357-4726-9903-975ae919e362 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:20.249460 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa2a1cc-125f-4df5-bdb0-cf3f00d38634 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:20.292150 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:20.307884 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a234795-e2ce-48f2-b95b-a1d7f3ce4d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:20.308905 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a234795-e2ce-48f2-b95b-a1d7f3ce4d4a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:20.309106 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a234795-e2ce-48f2-b95b-a1d7f3ce4d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:20.309473 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a234795-e2ce-48f2-b95b-a1d7f3ce4d4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:20.323553 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a234795-e2ce-48f2-b95b-a1d7f3ce4d4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:20.326398 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a234795-e2ce-48f2-b95b-a1d7f3ce4d4a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:20.339661 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c382db5f-f357-4726-9903-975ae919e362 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:20.346876 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-6a234795-e2ce-48f2-b95b-a1d7f3ce4d4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:20.346876 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:20.346876 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:20.346876 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:20.346876 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:20.346876 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:20.346876 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:20.346876 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:20.346876 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:20.346876 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:20.346876 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:20.346876 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:20.346876 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:54:20.346876 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:54:20.346876 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:54:20.348703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:54:20.348703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:20.348703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:20.348703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:20.348703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:20.348703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:20.348703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:20.348703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:20.348703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:20.348703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:20.348703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:20.348703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:20.348703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:20.348703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:20.348703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:20.348703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:20.348703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:20.350425 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:54:20.350425 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:54:20.350425 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:54:20.350425 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:54:20.350425 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:54:20.350425 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:54:20.350425 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:20.350425 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:20.350425 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 339/679] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:54:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 448 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:20.351409 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a234795-e2ce-48f2-b95b-a1d7f3ce4d4a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:20.391434 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3c438-31c1-4283-92e6-cf5722665d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:20.392228 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3c438-31c1-4283-92e6-cf5722665d2d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:20.392677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3c438-31c1-4283-92e6-cf5722665d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:20.393125 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3c438-31c1-4283-92e6-cf5722665d2d None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:20.393991 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f3c438-31c1-4283-92e6-cf5722665d2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:20.394738 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f3c438-31c1-4283-92e6-cf5722665d2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:20.396788 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f3c438-31c1-4283-92e6-cf5722665d2d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:20.400019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:20.421578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 341/680] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:20.421578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3c438-31c1-4283-92e6-cf5722665d2d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:20.518828 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a234795-e2ce-48f2-b95b-a1d7f3ce4d4a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:20.595404 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d644e76-d9f9-4d5c-8e10-416274a69d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:20.596634 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d644e76-d9f9-4d5c-8e10-416274a69d69 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:20.597130 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d644e76-d9f9-4d5c-8e10-416274a69d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:20.597613 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d644e76-d9f9-4d5c-8e10-416274a69d69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:20.624458 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d644e76-d9f9-4d5c-8e10-416274a69d69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:54:20.631519 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3c438-31c1-4283-92e6-cf5722665d2d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:20.634498 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-7d644e76-d9f9-4d5c-8e10-416274a69d69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:54:20.634498 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:20.634498 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:20.634498 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:20.634498 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:20.634498 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:20.634498 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:20.634498 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:20.634498 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:20.634498 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:20.634498 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:20.634498 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:20.634498 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:54:20.634498 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:54:20.634498 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:54:20.634498 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:54:20.636475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:54:20.636475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:54:20.636475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:54:20.636475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:54:20.636475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:20.636475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:20.636475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:20.636475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:20.636475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:20.636475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:20.636475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:20.636475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:20.636475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:20.636475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:20.636475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:20.636475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:20.638191 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:20.638191 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:20.638191 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:20.638191 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:20.638191 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:54:20.638191 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:54:20.638191 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:54:20.638191 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:54:20.638191 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:54:20.638191 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:54:20.638191 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:54:20.638191 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:20.638191 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 340/681] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:54:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:20.639951 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d644e76-d9f9-4d5c-8e10-416274a69d69 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:20.721788 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173cd6a0-744d-4b26-af79-073687513a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:20.723794 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173cd6a0-744d-4b26-af79-073687513a2e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:20.725537 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173cd6a0-744d-4b26-af79-073687513a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:20.726441 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173cd6a0-744d-4b26-af79-073687513a2e None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:20.728759 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173cd6a0-744d-4b26-af79-073687513a2e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:20.730131 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:20.730857 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173cd6a0-744d-4b26-af79-073687513a2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:20.735309 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173cd6a0-744d-4b26-af79-073687513a2e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:20.758927 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 342/682] 10.222.0.45 () {56 vars in 1135 bytes} [Mon Aug 1 13:54:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:20.760122 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173cd6a0-744d-4b26-af79-073687513a2e None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:20.847684 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:20.852640 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d644e76-d9f9-4d5c-8e10-416274a69d69 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:20.913565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173cd6a0-744d-4b26-af79-073687513a2e None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:20.924044 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215380b3-706a-4b78-a2ce-eb9ac5b4f03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:20.925170 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215380b3-706a-4b78-a2ce-eb9ac5b4f03b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:20.925664 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215380b3-706a-4b78-a2ce-eb9ac5b4f03b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:20.926150 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215380b3-706a-4b78-a2ce-eb9ac5b4f03b None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:20.927605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215380b3-706a-4b78-a2ce-eb9ac5b4f03b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:20.928169 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215380b3-706a-4b78-a2ce-eb9ac5b4f03b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:20.931009 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215380b3-706a-4b78-a2ce-eb9ac5b4f03b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:20.955060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 341/683] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:54:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:20.957048 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215380b3-706a-4b78-a2ce-eb9ac5b4f03b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:20.998620 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b48656-0aff-4374-8ad8-3ec5b029dc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:20.999590 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b48656-0aff-4374-8ad8-3ec5b029dc3c None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:21.000046 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b48656-0aff-4374-8ad8-3ec5b029dc3c None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:21.000520 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b48656-0aff-4374-8ad8-3ec5b029dc3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:21.012861 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b48656-0aff-4374-8ad8-3ec5b029dc3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:21.015707 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b48656-0aff-4374-8ad8-3ec5b029dc3c None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:21.044950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-49b48656-0aff-4374-8ad8-3ec5b029dc3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:21.044950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:21.044950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:21.044950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:21.044950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:21.044950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:21.044950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:21.044950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:21.044950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:21.044950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:21.044950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:21.044950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:21.044950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:54:21.044950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:54:21.044950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:54:21.046905 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:54:21.046905 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:21.046905 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:21.046905 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:21.046905 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:21.046905 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:21.046905 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:21.046905 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:21.046905 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:21.046905 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:21.046905 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:21.046905 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:21.046905 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:21.046905 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:21.046905 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:21.046905 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:21.046905 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:21.048766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:54:21.048766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:54:21.048766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:54:21.048766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:54:21.048766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:54:21.048766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:54:21.048766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:21.048766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:21.051121 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 343/684] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:54:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:21.054475 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b48656-0aff-4374-8ad8-3ec5b029dc3c None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:21.188871 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215380b3-706a-4b78-a2ce-eb9ac5b4f03b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:21.207260 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:21.257201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bca9db-7887-4e3e-9705-db3abf7ffe37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:21.258162 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bca9db-7887-4e3e-9705-db3abf7ffe37 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:21.258651 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bca9db-7887-4e3e-9705-db3abf7ffe37 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:21.259310 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bca9db-7887-4e3e-9705-db3abf7ffe37 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:21.260689 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bca9db-7887-4e3e-9705-db3abf7ffe37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:21.261242 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bca9db-7887-4e3e-9705-db3abf7ffe37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:21.263949 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bca9db-7887-4e3e-9705-db3abf7ffe37 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:21.265064 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b48656-0aff-4374-8ad8-3ec5b029dc3c None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:21.288696 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 342/685] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:54:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:21.291504 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bca9db-7887-4e3e-9705-db3abf7ffe37 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:21.297836 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:21.361411 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043bf275-42d2-45ff-94d6-fa63c78aec6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:21.362540 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043bf275-42d2-45ff-94d6-fa63c78aec6d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:21.363017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043bf275-42d2-45ff-94d6-fa63c78aec6d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:21.363494 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043bf275-42d2-45ff-94d6-fa63c78aec6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:21.504110 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bca9db-7887-4e3e-9705-db3abf7ffe37 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:21.505116 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043bf275-42d2-45ff-94d6-fa63c78aec6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:21.506973 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043bf275-42d2-45ff-94d6-fa63c78aec6d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:21.579013 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a31c213-1d15-4a6a-b27a-db3fa244b948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:21.580049 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31c213-1d15-4a6a-b27a-db3fa244b948 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:21.580575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31c213-1d15-4a6a-b27a-db3fa244b948 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:21.581050 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31c213-1d15-4a6a-b27a-db3fa244b948 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:21.582608 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a31c213-1d15-4a6a-b27a-db3fa244b948 None admin] RBAC: Adding query filter params (role.id=d17972f7fe97462b8f9f08aed9bc9cb5, scope.project.id=36f063d8f65d494a89dbbd72c53417c8, user.id=e7317c8829cb4fa1b30711f1db575cab) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:21.583169 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a31c213-1d15-4a6a-b27a-db3fa244b948 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:21.585946 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a31c213-1d15-4a6a-b27a-db3fa244b948 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:21.598240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 343/686] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:54:21 2022] GET /identity/v3/role_assignments?user.id=e7317c8829cb4fa1b30711f1db575cab&scope.project.id=36f063d8f65d494a89dbbd72c53417c8&role.id=d17972f7fe97462b8f9f08aed9bc9cb5 => generated 608 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:21.599807 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a31c213-1d15-4a6a-b27a-db3fa244b948 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:21.674945 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:21.748600 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 344/687] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:54:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 697 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:54:21.750199 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043bf275-42d2-45ff-94d6-fa63c78aec6d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:21.795269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:21.816529 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a31c213-1d15-4a6a-b27a-db3fa244b948 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:21.890663 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff0123-3570-46e9-b975-1488e64b5306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:21.892316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff0123-3570-46e9-b975-1488e64b5306 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:21.892954 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff0123-3570-46e9-b975-1488e64b5306 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:21.893446 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff0123-3570-46e9-b975-1488e64b5306 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:21.895195 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff0123-3570-46e9-b975-1488e64b5306 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:21.895739 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff0123-3570-46e9-b975-1488e64b5306 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:21.899589 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff0123-3570-46e9-b975-1488e64b5306 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:21.934481 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 344/688] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:54:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:21.934754 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff0123-3570-46e9-b975-1488e64b5306 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:21.990851 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043bf275-42d2-45ff-94d6-fa63c78aec6d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:22.065570 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9b94a0-6f73-429f-b1e8-80415176ca69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:22.066875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b94a0-6f73-429f-b1e8-80415176ca69 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:22.067257 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b94a0-6f73-429f-b1e8-80415176ca69 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:22.067846 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b94a0-6f73-429f-b1e8-80415176ca69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:22.089681 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9b94a0-6f73-429f-b1e8-80415176ca69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:22.092711 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9b94a0-6f73-429f-b1e8-80415176ca69 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:22.121925 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-ec9b94a0-6f73-429f-b1e8-80415176ca69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:22.121925 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:22.121925 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:22.121925 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:22.121925 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:22.121925 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:22.121925 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:22.121925 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:22.121925 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:22.121925 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:22.121925 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:22.121925 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:22.121925 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:22.121925 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:22.121925 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:22.123576 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:22.123576 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:22.123576 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:22.123576 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:22.123576 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:22.123576 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:22.123576 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:22.123576 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:22.123576 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:22.123576 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:22.123576 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:22.123576 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:22.123576 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:22.123576 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:22.123576 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:22.123576 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:22.123576 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:22.129733 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:22.129733 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:22.129733 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:22.129733 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:22.130195 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:22.130341 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 345/689] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:22.130924 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9b94a0-6f73-429f-b1e8-80415176ca69 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:22.145889 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff0123-3570-46e9-b975-1488e64b5306 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:22.217473 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f405d6-683a-4150-97ee-e514cd060fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:22.218659 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f405d6-683a-4150-97ee-e514cd060fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:22.219169 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f405d6-683a-4150-97ee-e514cd060fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:22.219664 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f405d6-683a-4150-97ee-e514cd060fe3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:22.225266 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:22.242628 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f405d6-683a-4150-97ee-e514cd060fe3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:22.245699 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f405d6-683a-4150-97ee-e514cd060fe3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:22.276519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-a8f405d6-683a-4150-97ee-e514cd060fe3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:22.276519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:22.276519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:22.276519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:22.276519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:22.276519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:22.276519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:22.276519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:22.276519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:22.276519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:22.276519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:22.276519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:22.276519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:22.276519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:22.276519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:22.278423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:22.278423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:22.278423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:22.278423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:22.278423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:22.278423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:22.278423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:22.278423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:22.278423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:22.278423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:22.278423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:22.278423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:22.278423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:22.278423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:22.278423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:22.278423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:22.278423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:22.280326 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:22.280326 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:22.280326 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:22.280326 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:22.284733 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 345/690] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:22.286780 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab88da-8c32-4761-bfb3-23da667935c4 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:22.287244 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab88da-8c32-4761-bfb3-23da667935c4 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:22.287672 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab88da-8c32-4761-bfb3-23da667935c4 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:22.290288 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 346/691] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:22.292317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab88da-8c32-4761-bfb3-23da667935c4 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:22.306740 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9b94a0-6f73-429f-b1e8-80415176ca69 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:22.376005 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ff7a2-b734-4db2-b21e-75b083344966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:22.377221 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ff7a2-b734-4db2-b21e-75b083344966 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:22.377694 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ff7a2-b734-4db2-b21e-75b083344966 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:22.378173 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ff7a2-b734-4db2-b21e-75b083344966 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:22.379855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449ff7a2-b734-4db2-b21e-75b083344966 None admin] RBAC: Adding query filter params (role.id=d17972f7fe97462b8f9f08aed9bc9cb5, scope.project.id=36f063d8f65d494a89dbbd72c53417c8, user.id=6dd44b7a78594288b6aabb393fd6324a) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:22.380438 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449ff7a2-b734-4db2-b21e-75b083344966 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:22.383372 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449ff7a2-b734-4db2-b21e-75b083344966 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:22.404331 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 346/692] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:54:22 2022] GET /identity/v3/role_assignments?user.id=6dd44b7a78594288b6aabb393fd6324a&scope.project.id=36f063d8f65d494a89dbbd72c53417c8&role.id=d17972f7fe97462b8f9f08aed9bc9cb5 => generated 608 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:22.404331 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ff7a2-b734-4db2-b21e-75b083344966 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:22.529921 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab88da-8c32-4761-bfb3-23da667935c4 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:22.616339 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a777d8-818f-4fc9-8787-74d6fd02236d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:22.617505 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a777d8-818f-4fc9-8787-74d6fd02236d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:22.618001 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a777d8-818f-4fc9-8787-74d6fd02236d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:22.618557 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a777d8-818f-4fc9-8787-74d6fd02236d None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:22.620006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a777d8-818f-4fc9-8787-74d6fd02236d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:22.620575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a777d8-818f-4fc9-8787-74d6fd02236d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:22.623413 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a777d8-818f-4fc9-8787-74d6fd02236d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:22.646003 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 347/693] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:22.647884 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a81b5e-26ec-4eba-acb8-4c2df5226a88 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:22.648430 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a81b5e-26ec-4eba-acb8-4c2df5226a88 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:22.648815 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a81b5e-26ec-4eba-acb8-4c2df5226a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:22.652733 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ff7a2-b734-4db2-b21e-75b083344966 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:22.671636 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:22.685918 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-53a81b5e-26ec-4eba-acb8-4c2df5226a88 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:22.686385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-53a81b5e-26ec-4eba-acb8-4c2df5226a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:22.717304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:22.730122 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80017c75-eecf-4d3a-94da-71b45bb745ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:22.731732 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80017c75-eecf-4d3a-94da-71b45bb745ec None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:22.732490 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80017c75-eecf-4d3a-94da-71b45bb745ec None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:22.733262 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80017c75-eecf-4d3a-94da-71b45bb745ec None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:22.734820 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80017c75-eecf-4d3a-94da-71b45bb745ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:22.735554 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80017c75-eecf-4d3a-94da-71b45bb745ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:22.744111 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80017c75-eecf-4d3a-94da-71b45bb745ec None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:22.769814 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 347/694] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:22.771724 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80017c75-eecf-4d3a-94da-71b45bb745ec None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:22.836407 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 348/695] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:54:22.838160 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a81b5e-26ec-4eba-acb8-4c2df5226a88 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:22.982292 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80017c75-eecf-4d3a-94da-71b45bb745ec None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:23.054148 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132aaaf1-c747-4486-8424-22ff2c7601fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:23.055893 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132aaaf1-c747-4486-8424-22ff2c7601fb None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:23.056590 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132aaaf1-c747-4486-8424-22ff2c7601fb None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:23.057259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132aaaf1-c747-4486-8424-22ff2c7601fb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:23.071006 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132aaaf1-c747-4486-8424-22ff2c7601fb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:23.073064 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a81b5e-26ec-4eba-acb8-4c2df5226a88 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:23.073407 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132aaaf1-c747-4486-8424-22ff2c7601fb None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:23.088552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-132aaaf1-c747-4486-8424-22ff2c7601fb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:54:23.088552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:23.088552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:23.088552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:23.088552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:23.088552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:23.088552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:23.088552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:23.088552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:23.088552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:23.088552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:23.088552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:23.088552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:54:23.088552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:54:23.088552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:54:23.088552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:54:23.089810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:23.089810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:23.089810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:54:23.089810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:23.089810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:54:23.089810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:23.089810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:23.089810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:23.089810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:23.089810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:23.089810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:23.089810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:23.089810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:23.089810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:23.089810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:23.089810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:23.089810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:23.089810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:23.090874 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:23.090874 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:23.090874 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:54:23.090874 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:54:23.090874 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:54:23.090874 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:54:23.090874 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:54:23.090874 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:54:23.090874 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:54:23.090874 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:23.093421 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 348/696] 10.222.0.45 () {56 vars in 1093 bytes} [Mon Aug 1 13:54:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:23.094682 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132aaaf1-c747-4486-8424-22ff2c7601fb None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:23.143116 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664fb5ab-e4ad-4fbc-8bc2-4aeec458ddd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:23.144208 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664fb5ab-e4ad-4fbc-8bc2-4aeec458ddd6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:23.145037 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664fb5ab-e4ad-4fbc-8bc2-4aeec458ddd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:23.145516 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664fb5ab-e4ad-4fbc-8bc2-4aeec458ddd6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:23.150776 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:23.159931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:23.161342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664fb5ab-e4ad-4fbc-8bc2-4aeec458ddd6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:23.164982 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664fb5ab-e4ad-4fbc-8bc2-4aeec458ddd6 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:23.199114 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-664fb5ab-e4ad-4fbc-8bc2-4aeec458ddd6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:23.199114 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:23.199114 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:23.199114 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:23.199114 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:23.199114 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:23.199114 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:23.199114 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:23.199114 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:23.199114 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:23.199114 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:23.199114 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:23.199114 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:54:23.199114 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:54:23.199114 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:54:23.201097 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:54:23.201097 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:23.201097 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:23.201097 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:23.201097 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:23.201097 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:23.201097 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:23.201097 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:23.201097 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:23.201097 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:23.201097 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:23.201097 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:23.201097 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:23.201097 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:23.201097 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:23.201097 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:23.201097 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:23.203174 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:54:23.203174 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:54:23.203174 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:54:23.203174 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:54:23.203174 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:54:23.203174 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:54:23.203174 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:23.203174 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:23.212686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 349/697] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:54:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:23.214741 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664fb5ab-e4ad-4fbc-8bc2-4aeec458ddd6 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:23.242585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132aaaf1-c747-4486-8424-22ff2c7601fb None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:23.284207 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eedd0b-b555-4b92-b2c7-9d50f1f5e130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:23.285090 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eedd0b-b555-4b92-b2c7-9d50f1f5e130 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:23.285481 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eedd0b-b555-4b92-b2c7-9d50f1f5e130 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:23.287052 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eedd0b-b555-4b92-b2c7-9d50f1f5e130 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:23.287052 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eedd0b-b555-4b92-b2c7-9d50f1f5e130 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:23.288194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eedd0b-b555-4b92-b2c7-9d50f1f5e130 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:23.301080 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-65eedd0b-b555-4b92-b2c7-9d50f1f5e130 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 13:54:23.301080 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:23.301080 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:54:23.301080 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:54:23.301080 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:54:23.301080 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:54:23.301080 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:54:23.301080 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:54:23.301080 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 13:54:23.301080 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:23.301080 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:54:23.301080 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:23.301080 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:23.301080 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:23.301080 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:23.301080 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:23.303076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:23.303076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:23.303076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:23.303076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:23.303076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:23.303076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:23.303076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:23.303076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:54:23.303076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:54:23.303076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:54:23.303076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:54:23.303076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:23.303076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:23.303076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:23.303076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:23.303076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:23.304880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:23.304880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:23.304880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:23.304880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:23.304880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:23.304880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:23.304880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:23.304880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:23.304880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:23.304880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:23.304880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:23.304880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:54:23.304880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:54:23.304880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 13:54:23.304880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:23.311296 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 349/698] 10.222.0.45 () {56 vars in 1129 bytes} [Mon Aug 1 13:54:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:23.312436 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eedd0b-b555-4b92-b2c7-9d50f1f5e130 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:23.431408 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664fb5ab-e4ad-4fbc-8bc2-4aeec458ddd6 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:23.491024 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eedd0b-b555-4b92-b2c7-9d50f1f5e130 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:23.505313 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092cf9c-0dfe-44cc-9a4e-325543be1f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:23.506153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092cf9c-0dfe-44cc-9a4e-325543be1f03 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:23.506514 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092cf9c-0dfe-44cc-9a4e-325543be1f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:23.506856 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092cf9c-0dfe-44cc-9a4e-325543be1f03 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:23.508973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7092cf9c-0dfe-44cc-9a4e-325543be1f03 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:23.509377 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7092cf9c-0dfe-44cc-9a4e-325543be1f03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:23.510938 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7092cf9c-0dfe-44cc-9a4e-325543be1f03 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:23.523565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:23.556151 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 350/699] 10.222.0.45 () {56 vars in 1133 bytes} [Mon Aug 1 13:54:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:23.557827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092cf9c-0dfe-44cc-9a4e-325543be1f03 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:23.585307 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44f000-048d-4d07-85ca-e63e84ea95bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:23.586816 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44f000-048d-4d07-85ca-e63e84ea95bd None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:23.587569 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44f000-048d-4d07-85ca-e63e84ea95bd None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:23.588325 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44f000-048d-4d07-85ca-e63e84ea95bd None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:23.590079 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c44f000-048d-4d07-85ca-e63e84ea95bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:23.590883 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c44f000-048d-4d07-85ca-e63e84ea95bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:23.594091 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c44f000-048d-4d07-85ca-e63e84ea95bd None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:23.617374 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 350/700] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:54:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:23.619335 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44f000-048d-4d07-85ca-e63e84ea95bd None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:23.669886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:23.763751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092cf9c-0dfe-44cc-9a4e-325543be1f03 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:23.822847 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44f000-048d-4d07-85ca-e63e84ea95bd None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:23.836895 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df1083-cc64-43d6-b903-42f52c3d3dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:23.837992 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df1083-cc64-43d6-b903-42f52c3d3dfe None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:23.838506 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df1083-cc64-43d6-b903-42f52c3d3dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:23.839064 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df1083-cc64-43d6-b903-42f52c3d3dfe None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:23.840759 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df1083-cc64-43d6-b903-42f52c3d3dfe None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:23.841262 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df1083-cc64-43d6-b903-42f52c3d3dfe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:23.843952 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df1083-cc64-43d6-b903-42f52c3d3dfe None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:23.853330 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 351/701] 10.222.0.45 () {56 vars in 1124 bytes} [Mon Aug 1 13:54:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:23.855133 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df1083-cc64-43d6-b903-42f52c3d3dfe None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:23.883426 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560977a1-1320-49af-abfd-3d005b8ac555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:23.884802 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560977a1-1320-49af-abfd-3d005b8ac555 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:23.885397 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560977a1-1320-49af-abfd-3d005b8ac555 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:23.885879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560977a1-1320-49af-abfd-3d005b8ac555 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:23.942324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:23.977082 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560977a1-1320-49af-abfd-3d005b8ac555 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:23.979058 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560977a1-1320-49af-abfd-3d005b8ac555 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:24.064874 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df1083-cc64-43d6-b903-42f52c3d3dfe None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:24.091439 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:24.119193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b930e43d-9dc1-44ab-b3af-d3d740f1af2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:24.120799 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930e43d-9dc1-44ab-b3af-d3d740f1af2c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:24.121666 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930e43d-9dc1-44ab-b3af-d3d740f1af2c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:24.122301 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930e43d-9dc1-44ab-b3af-d3d740f1af2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:24.123378 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b930e43d-9dc1-44ab-b3af-d3d740f1af2c None admin] RBAC: Adding query filter params (role.id=d17972f7fe97462b8f9f08aed9bc9cb5, scope.project.id=36f063d8f65d494a89dbbd72c53417c8, user.id=0cda7f365ba0405daaf86870960f12d2) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:24.123952 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b930e43d-9dc1-44ab-b3af-d3d740f1af2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:24.126051 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b930e43d-9dc1-44ab-b3af-d3d740f1af2c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:24.138603 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 352/702] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:54:23 2022] GET /identity/v3/role_assignments?user.id=0cda7f365ba0405daaf86870960f12d2&scope.project.id=36f063d8f65d494a89dbbd72c53417c8&role.id=d17972f7fe97462b8f9f08aed9bc9cb5 => generated 608 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:24.139921 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b930e43d-9dc1-44ab-b3af-d3d740f1af2c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:24.207913 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 351/703] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:54:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:54:24.216929 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560977a1-1320-49af-abfd-3d005b8ac555 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:24.427785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b930e43d-9dc1-44ab-b3af-d3d740f1af2c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:24.456108 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:24.516803 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560977a1-1320-49af-abfd-3d005b8ac555 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:24.546760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3c78d-4ffe-49b2-8ff5-9607c089f247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:24.547922 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3c78d-4ffe-49b2-8ff5-9607c089f247 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:24.548475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3c78d-4ffe-49b2-8ff5-9607c089f247 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:24.548995 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3c78d-4ffe-49b2-8ff5-9607c089f247 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:24.550328 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba3c78d-4ffe-49b2-8ff5-9607c089f247 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:24.550882 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba3c78d-4ffe-49b2-8ff5-9607c089f247 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:24.553784 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba3c78d-4ffe-49b2-8ff5-9607c089f247 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:24.565972 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 353/704] 10.222.0.45 () {56 vars in 1124 bytes} [Mon Aug 1 13:54:24 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 426 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:24.607010 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c81f0a-f48c-4da0-9177-d4857c0e57f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:24.608250 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c81f0a-f48c-4da0-9177-d4857c0e57f8 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:24.609034 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c81f0a-f48c-4da0-9177-d4857c0e57f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:24.609574 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c81f0a-f48c-4da0-9177-d4857c0e57f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:24.627012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c81f0a-f48c-4da0-9177-d4857c0e57f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:24.629174 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c81f0a-f48c-4da0-9177-d4857c0e57f8 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:24.664663 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-a9c81f0a-f48c-4da0-9177-d4857c0e57f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:24.664663 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:24.664663 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:24.664663 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:24.664663 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:24.664663 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:24.664663 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:24.664663 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:24.664663 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:24.664663 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:24.664663 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:24.664663 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:24.664663 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:24.664663 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:24.664663 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:24.666589 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:24.666589 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:24.666589 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:24.666589 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:24.666589 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:24.666589 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:24.666589 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:24.666589 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:24.666589 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:24.666589 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:24.666589 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:24.666589 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:24.666589 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:24.666589 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:24.666589 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:24.666589 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:24.666589 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:24.668612 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:24.668612 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:24.668612 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:24.668612 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:24.673212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 352/705] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 458 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:24.681273 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3c78d-4ffe-49b2-8ff5-9607c089f247 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:24.793613 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:24.927096 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3c78d-4ffe-49b2-8ff5-9607c089f247 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:25.003990 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05d6b2-a15c-4871-ab86-b43b91c2747d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:25.005714 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05d6b2-a15c-4871-ab86-b43b91c2747d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:25.006640 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05d6b2-a15c-4871-ab86-b43b91c2747d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:25.007431 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05d6b2-a15c-4871-ab86-b43b91c2747d None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:25.009150 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da05d6b2-a15c-4871-ab86-b43b91c2747d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:25.010053 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da05d6b2-a15c-4871-ab86-b43b91c2747d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:25.013201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da05d6b2-a15c-4871-ab86-b43b91c2747d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:25.042792 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9927a99a-f67c-426b-a796-62736cd5ef3f None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:25.043787 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9927a99a-f67c-426b-a796-62736cd5ef3f None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:25.044657 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9927a99a-f67c-426b-a796-62736cd5ef3f None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:25.047900 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 353/707] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:25.048145 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 354/707] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:25.049350 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05d6b2-a15c-4871-ab86-b43b91c2747d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:25.059196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72243e-6173-49d2-bc58-495ac6eda5df None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:25.059732 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72243e-6173-49d2-bc58-495ac6eda5df None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:25.060131 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72243e-6173-49d2-bc58-495ac6eda5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:25.122014 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-6a72243e-6173-49d2-bc58-495ac6eda5df None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:25.123907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-6a72243e-6173-49d2-bc58-495ac6eda5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:25.322360 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05d6b2-a15c-4871-ab86-b43b91c2747d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:25.335285 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:25.349111 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 354/708] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:54:25.351742 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467c808-d4d8-4149-a2a1-c6f4ca4a56b3 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:25.352470 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467c808-d4d8-4149-a2a1-c6f4ca4a56b3 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:25.352920 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467c808-d4d8-4149-a2a1-c6f4ca4a56b3 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:25.360417 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 355/709] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:25.364536 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a02bd0-3851-4754-8750-0d55ed5f0e97 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:25.365371 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a02bd0-3851-4754-8750-0d55ed5f0e97 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:25.366015 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a02bd0-3851-4754-8750-0d55ed5f0e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:25.398586 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-96a02bd0-3851-4754-8750-0d55ed5f0e97 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:25.401149 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-96a02bd0-3851-4754-8750-0d55ed5f0e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:25.405053 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f028ed02-4f2f-4adf-bb98-61cb3cddc6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:25.406299 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f028ed02-4f2f-4adf-bb98-61cb3cddc6e2 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:25.406705 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f028ed02-4f2f-4adf-bb98-61cb3cddc6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:25.407183 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f028ed02-4f2f-4adf-bb98-61cb3cddc6e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:25.419989 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f028ed02-4f2f-4adf-bb98-61cb3cddc6e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:25.421838 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f028ed02-4f2f-4adf-bb98-61cb3cddc6e2 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:25.439510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-f028ed02-4f2f-4adf-bb98-61cb3cddc6e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:25.439510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:25.439510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:25.439510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:25.439510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:25.439510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:25.439510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:25.439510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:25.439510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:25.439510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:25.439510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:25.439510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:25.439510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:54:25.439510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:54:25.439510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:54:25.441949 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:54:25.441949 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:25.441949 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:25.441949 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:25.441949 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:25.441949 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:25.441949 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:25.441949 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:25.441949 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:25.441949 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:25.441949 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:25.441949 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:25.441949 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:25.441949 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:25.441949 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:25.441949 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:25.441949 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:25.444444 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:54:25.444444 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:54:25.444444 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:54:25.444444 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:54:25.444444 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:54:25.444444 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:54:25.444444 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:25.444444 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:25.444444 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 355/710] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:54:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:25.444444 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f028ed02-4f2f-4adf-bb98-61cb3cddc6e2 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:25.515707 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:25.557516 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 356/711] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:54:25.560422 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a02bd0-3851-4754-8750-0d55ed5f0e97 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:25.707810 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f028ed02-4f2f-4adf-bb98-61cb3cddc6e2 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:25.784190 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4fc0c6-a4e5-4688-8430-71a74ff223a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:25.785832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4fc0c6-a4e5-4688-8430-71a74ff223a6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:25.786656 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4fc0c6-a4e5-4688-8430-71a74ff223a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:25.787110 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a02bd0-3851-4754-8750-0d55ed5f0e97 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:25.788024 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4fc0c6-a4e5-4688-8430-71a74ff223a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:25.812753 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4fc0c6-a4e5-4688-8430-71a74ff223a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:25.816260 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4fc0c6-a4e5-4688-8430-71a74ff223a6 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:25.848962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-ec4fc0c6-a4e5-4688-8430-71a74ff223a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:25.848962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:25.848962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:25.848962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:25.848962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:25.848962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:25.848962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:25.848962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:25.848962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:25.848962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:25.848962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:25.848962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:25.848962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:25.848962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:25.848962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:25.850693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:25.850693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:25.850693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:25.850693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:25.850693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:25.850693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:25.850693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:25.850693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:25.850693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:25.850693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:25.850693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:25.850693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:25.850693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:25.850693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:25.850693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:25.850693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:25.850693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:25.852365 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:25.852365 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:25.852365 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:25.852365 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:25.856246 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 356/712] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 414 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:25.858320 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4fc0c6-a4e5-4688-8430-71a74ff223a6 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:25.867236 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cbfee-c9a7-4258-8dbf-71dc0317c8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:25.867680 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cbfee-c9a7-4258-8dbf-71dc0317c8cb None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:25.868265 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cbfee-c9a7-4258-8dbf-71dc0317c8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:25.868880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cbfee-c9a7-4258-8dbf-71dc0317c8cb None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:25.870316 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069cbfee-c9a7-4258-8dbf-71dc0317c8cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:25.871037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069cbfee-c9a7-4258-8dbf-71dc0317c8cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:25.873760 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069cbfee-c9a7-4258-8dbf-71dc0317c8cb None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:25.896046 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 357/713] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:54:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:25.897619 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cbfee-c9a7-4258-8dbf-71dc0317c8cb None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:25.906198 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:25.983880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:26.080939 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cbfee-c9a7-4258-8dbf-71dc0317c8cb None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:26.089866 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4fc0c6-a4e5-4688-8430-71a74ff223a6 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:26.142770 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c579c3bc-9acf-4599-b40c-34ee45e250c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:26.143852 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c579c3bc-9acf-4599-b40c-34ee45e250c7 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:26.144724 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c579c3bc-9acf-4599-b40c-34ee45e250c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:26.145238 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c579c3bc-9acf-4599-b40c-34ee45e250c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:26.146509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c579c3bc-9acf-4599-b40c-34ee45e250c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:26.147089 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c579c3bc-9acf-4599-b40c-34ee45e250c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:26.149570 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c579c3bc-9acf-4599-b40c-34ee45e250c7 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:26.167065 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45acef13-be2d-4b18-9fb0-da2f648a1063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:26.168534 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45acef13-be2d-4b18-9fb0-da2f648a1063 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:26.169234 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45acef13-be2d-4b18-9fb0-da2f648a1063 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:26.169918 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45acef13-be2d-4b18-9fb0-da2f648a1063 None admin] PATH_INFO: `/v3/services/network` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:26.171503 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45acef13-be2d-4b18-9fb0-da2f648a1063 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:26.174435 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45acef13-be2d-4b18-9fb0-da2f648a1063 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:26.176790 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 358/714] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:26.178271 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c579c3bc-9acf-4599-b40c-34ee45e250c7 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:26.195832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-45acef13-be2d-4b18-9fb0-da2f648a1063 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 13:54:26.195832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:26.195832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:54:26.195832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:54:26.195832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:54:26.195832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:54:26.195832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:54:26.195832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:54:26.195832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 13:54:26.195832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:26.195832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:54:26.195832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:26.195832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:26.195832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:26.195832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:26.195832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:26.195832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:26.197836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:26.197836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:26.197836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:26.197836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:26.197836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:26.197836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:26.197836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:54:26.197836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:54:26.197836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:54:26.197836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:54:26.197836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:26.197836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:26.197836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:26.197836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:26.197836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:26.197836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:26.199528 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:26.199528 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:26.199528 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:26.199528 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:26.199528 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:26.199528 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:26.199528 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:26.199528 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:26.199528 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:26.199528 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:26.199528 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:54:26.199528 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:54:26.199528 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 13:54:26.199528 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:26.205553 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 357/715] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:54:25 2022] GET /identity/v3/services/network => generated 88 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:26.214405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45acef13-be2d-4b18-9fb0-da2f648a1063 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:26.361604 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c579c3bc-9acf-4599-b40c-34ee45e250c7 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:26.378459 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:26.408912 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:26.422870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45acef13-be2d-4b18-9fb0-da2f648a1063 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:26.434803 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5081d-aa6a-4f56-9f86-a210f5b099ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:26.435709 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5081d-aa6a-4f56-9f86-a210f5b099ac None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:26.436120 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5081d-aa6a-4f56-9f86-a210f5b099ac None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:26.436564 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5081d-aa6a-4f56-9f86-a210f5b099ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:26.437931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5081d-aa6a-4f56-9f86-a210f5b099ac None admin] RBAC: Adding query filter params (role.id=3e1a9ff016c840c5b51ef7e3da9900ac, scope.project.id=36f063d8f65d494a89dbbd72c53417c8, user.id=5b3adc14dcaf4ac79b43bb97fa1ba1d4) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:26.438391 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5081d-aa6a-4f56-9f86-a210f5b099ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:26.440809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5081d-aa6a-4f56-9f86-a210f5b099ac None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:26.451197 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 359/716] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:54:26 2022] GET /identity/v3/role_assignments?user.id=5b3adc14dcaf4ac79b43bb97fa1ba1d4&scope.project.id=36f063d8f65d494a89dbbd72c53417c8&role.id=3e1a9ff016c840c5b51ef7e3da9900ac => generated 259 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:26.452938 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5081d-aa6a-4f56-9f86-a210f5b099ac None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:26.512971 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14986c1-9767-4756-97e3-2e8fba16fa77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:26.514683 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14986c1-9767-4756-97e3-2e8fba16fa77 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:26.516298 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14986c1-9767-4756-97e3-2e8fba16fa77 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:26.518103 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14986c1-9767-4756-97e3-2e8fba16fa77 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:26.518686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14986c1-9767-4756-97e3-2e8fba16fa77 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:26.521534 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14986c1-9767-4756-97e3-2e8fba16fa77 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:26.592763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 358/717] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 13:54:26 2022] POST /identity/v3/users => generated 254 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:54:26.699655 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5081d-aa6a-4f56-9f86-a210f5b099ac None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:26.769720 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c6010c-7c43-4955-8b0d-4005e481cc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:26.771340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6010c-7c43-4955-8b0d-4005e481cc6a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:26.772162 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6010c-7c43-4955-8b0d-4005e481cc6a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:26.772988 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6010c-7c43-4955-8b0d-4005e481cc6a None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:26.774673 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c6010c-7c43-4955-8b0d-4005e481cc6a None admin] RBAC: Adding query filter params (name=network) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:26.775580 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c6010c-7c43-4955-8b0d-4005e481cc6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:26.778584 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c6010c-7c43-4955-8b0d-4005e481cc6a None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:26.788445 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 360/718] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Aug 1 13:54:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:26.796810 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14986c1-9767-4756-97e3-2e8fba16fa77 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:27.014413 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14986c1-9767-4756-97e3-2e8fba16fa77 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:27.083870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5d386-b958-418c-85d2-a0952f52c0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:27.084841 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5d386-b958-418c-85d2-a0952f52c0d5 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:27.085305 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5d386-b958-418c-85d2-a0952f52c0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:27.085764 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5d386-b958-418c-85d2-a0952f52c0d5 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:27.086964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5d386-b958-418c-85d2-a0952f52c0d5 None admin] RBAC: Adding query filter params (type=network) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:27.087293 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5d386-b958-418c-85d2-a0952f52c0d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:27.088776 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5d386-b958-418c-85d2-a0952f52c0d5 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:27.100907 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 359/719] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Aug 1 13:54:26 2022] GET /identity/v3/services?type=network => generated 124 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:27.588623 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3c7dd-aa2f-4ff2-8399-f800d25094b1 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:27.589511 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3c7dd-aa2f-4ff2-8399-f800d25094b1 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:27.590160 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3c7dd-aa2f-4ff2-8399-f800d25094b1 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:27.592719 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 361/720] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:27.599602 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979cac5-7174-49a9-8700-923c786bcec3 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:27.600098 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979cac5-7174-49a9-8700-923c786bcec3 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:27.600620 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979cac5-7174-49a9-8700-923c786bcec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:27.615247 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:27.642171 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-1979cac5-7174-49a9-8700-923c786bcec3 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:27.644322 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1979cac5-7174-49a9-8700-923c786bcec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:27.788840 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 360/721] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:54:27.830991 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a3c7dd-aa2f-4ff2-8399-f800d25094b1 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:27.884254 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:28.024553 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a3c7dd-aa2f-4ff2-8399-f800d25094b1 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:28.030747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cbde7f-2ced-47ef-a782-420bff521dcf None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:28.031579 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cbde7f-2ced-47ef-a782-420bff521dcf None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:28.031579 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cbde7f-2ced-47ef-a782-420bff521dcf None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:28.037468 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 361/722] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:54:28.045054 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294d2d4-2bdd-4ba6-a17a-0ad4f86b08d9 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:28.045486 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294d2d4-2bdd-4ba6-a17a-0ad4f86b08d9 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:28.045930 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294d2d4-2bdd-4ba6-a17a-0ad4f86b08d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:28.099088 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c376f3-6f09-49d7-8b2a-88a46393961c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:28.099922 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c376f3-6f09-49d7-8b2a-88a46393961c None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:28.100380 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c376f3-6f09-49d7-8b2a-88a46393961c None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:28.100795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c376f3-6f09-49d7-8b2a-88a46393961c None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:28.101991 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c376f3-6f09-49d7-8b2a-88a46393961c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:28.103650 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-6294d2d4-2bdd-4ba6-a17a-0ad4f86b08d9 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:28.104493 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c376f3-6f09-49d7-8b2a-88a46393961c None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:28.105618 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6294d2d4-2bdd-4ba6-a17a-0ad4f86b08d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:28.142339 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 362/723] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Aug 1 13:54:27 2022] POST /identity/v3/services => generated 251 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:54:28.237964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 362/724] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:28 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:54:28.275800 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c376f3-6f09-49d7-8b2a-88a46393961c None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:28.459307 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:28.471969 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c376f3-6f09-49d7-8b2a-88a46393961c None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:28.562649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55c884-a634-4820-ad87-3ad013c6ef1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:28.563709 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55c884-a634-4820-ad87-3ad013c6ef1f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:28.564172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55c884-a634-4820-ad87-3ad013c6ef1f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:28.564666 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55c884-a634-4820-ad87-3ad013c6ef1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:28.592872 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e55c884-a634-4820-ad87-3ad013c6ef1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:54:28.604302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-3e55c884-a634-4820-ad87-3ad013c6ef1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:54:28.604302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:28.604302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:28.604302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:28.604302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:28.604302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:28.604302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:28.604302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:28.604302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:28.604302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:28.604302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:28.604302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:28.604302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:54:28.604302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:54:28.604302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:54:28.604302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:54:28.606165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:54:28.606165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:54:28.606165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:54:28.606165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:54:28.606165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:28.606165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:28.606165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:28.606165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:28.606165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:28.606165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:28.606165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:28.606165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:28.606165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:28.606165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:28.606165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:28.606165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:28.607809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:28.607809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:28.607809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:28.607809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:28.607809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:54:28.607809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:54:28.607809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:54:28.607809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:54:28.607809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:54:28.607809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:54:28.607809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:54:28.607809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:28.613004 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 363/725] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:54:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:28.619681 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294d2d4-2bdd-4ba6-a17a-0ad4f86b08d9 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:28.739996 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:28.823578 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294d2d4-2bdd-4ba6-a17a-0ad4f86b08d9 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:28.900974 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da560845-1651-414a-a0fc-02b5b846b838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:28.902041 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da560845-1651-414a-a0fc-02b5b846b838 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:28.902536 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da560845-1651-414a-a0fc-02b5b846b838 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:28.903050 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da560845-1651-414a-a0fc-02b5b846b838 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:28.904332 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da560845-1651-414a-a0fc-02b5b846b838 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:28.904861 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da560845-1651-414a-a0fc-02b5b846b838 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:28.907459 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da560845-1651-414a-a0fc-02b5b846b838 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:28.926600 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 363/726] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:54:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:28.932557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55c884-a634-4820-ad87-3ad013c6ef1f None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:29.178678 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55c884-a634-4820-ad87-3ad013c6ef1f None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:29.240901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b88535f-f135-4471-b4fa-0a2a577a89de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:29.241947 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88535f-f135-4471-b4fa-0a2a577a89de None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:29.242432 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88535f-f135-4471-b4fa-0a2a577a89de None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:29.242902 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88535f-f135-4471-b4fa-0a2a577a89de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:29.263286 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b88535f-f135-4471-b4fa-0a2a577a89de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:29.266064 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b88535f-f135-4471-b4fa-0a2a577a89de None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:29.294525 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-5b88535f-f135-4471-b4fa-0a2a577a89de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:29.294525 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:29.294525 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:29.294525 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:29.294525 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:29.294525 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:29.294525 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:29.294525 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:29.294525 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:29.294525 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:29.294525 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:29.294525 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:29.294525 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:29.294525 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:29.294525 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:29.296340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:29.296340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:29.296340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:29.296340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:29.296340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:29.296340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:29.296340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:29.296340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:29.296340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:29.296340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:29.296340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:29.296340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:29.296340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:29.296340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:29.296340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:29.296340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:29.296340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:29.298212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:29.298212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:29.298212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:29.298212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:29.298212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 364/727] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:29.306505 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da560845-1651-414a-a0fc-02b5b846b838 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:29.572838 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da560845-1651-414a-a0fc-02b5b846b838 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:29.602623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:29.623475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec3aa3-65c3-4a96-a6bc-06a2d675d350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:29.624105 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec3aa3-65c3-4a96-a6bc-06a2d675d350 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:29.624411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec3aa3-65c3-4a96-a6bc-06a2d675d350 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:29.624742 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec3aa3-65c3-4a96-a6bc-06a2d675d350 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:29.625449 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ec3aa3-65c3-4a96-a6bc-06a2d675d350 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:29.625749 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ec3aa3-65c3-4a96-a6bc-06a2d675d350 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:29.627143 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ec3aa3-65c3-4a96-a6bc-06a2d675d350 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:29.646718 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 364/728] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:29.653507 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b88535f-f135-4471-b4fa-0a2a577a89de None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:29.677547 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:29.907629 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b88535f-f135-4471-b4fa-0a2a577a89de None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:29.990974 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b94138-8b9e-4a09-a304-9356b9e8a9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:29.993119 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b94138-8b9e-4a09-a304-9356b9e8a9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:29.993828 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b94138-8b9e-4a09-a304-9356b9e8a9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:29.994486 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b94138-8b9e-4a09-a304-9356b9e8a9f4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:30.012160 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b94138-8b9e-4a09-a304-9356b9e8a9f4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:30.014989 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b94138-8b9e-4a09-a304-9356b9e8a9f4 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:30.038305 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-58b94138-8b9e-4a09-a304-9356b9e8a9f4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 13:54:30.038305 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:30.038305 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:30.038305 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:30.038305 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:30.038305 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:30.038305 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:30.038305 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:30.038305 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:30.038305 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:30.038305 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:30.038305 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:30.038305 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:54:30.038305 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:54:30.038305 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:54:30.038305 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:54:30.039914 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:30.039914 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:30.039914 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:54:30.039914 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:30.039914 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:54:30.039914 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:30.039914 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:30.039914 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:30.039914 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:30.039914 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:30.039914 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:30.039914 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:30.039914 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:30.039914 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:30.039914 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:30.039914 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:30.039914 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:30.039914 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:30.041599 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:30.041599 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:30.041599 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:54:30.041599 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:54:30.041599 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:54:30.041599 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:54:30.041599 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:54:30.041599 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:54:30.041599 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 13:54:30.041599 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:30.045274 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 365/729] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:54:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:30.055221 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec3aa3-65c3-4a96-a6bc-06a2d675d350 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:30.313375 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec3aa3-65c3-4a96-a6bc-06a2d675d350 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:30.356461 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992bced-df87-40f3-b50b-57c3868d5979 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:30.356948 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992bced-df87-40f3-b50b-57c3868d5979 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:30.357423 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992bced-df87-40f3-b50b-57c3868d5979 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:30.365193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 366/730] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:30 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:54:30.373226 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a53ac-024c-4a86-8893-9a301c8d5c0a None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:30.373856 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a53ac-024c-4a86-8893-9a301c8d5c0a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:30.374397 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a53ac-024c-4a86-8893-9a301c8d5c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:30.390319 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32fdcf-90a3-4511-990c-85cb1a9f873c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:30.391411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32fdcf-90a3-4511-990c-85cb1a9f873c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:30.391884 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32fdcf-90a3-4511-990c-85cb1a9f873c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:30.392392 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32fdcf-90a3-4511-990c-85cb1a9f873c None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:30.393927 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f32fdcf-90a3-4511-990c-85cb1a9f873c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:30.394463 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f32fdcf-90a3-4511-990c-85cb1a9f873c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:30.397593 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f32fdcf-90a3-4511-990c-85cb1a9f873c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:30.420084 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-146a53ac-024c-4a86-8893-9a301c8d5c0a None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:30.425214 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-146a53ac-024c-4a86-8893-9a301c8d5c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:30.451924 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 365/731] 10.222.0.45 () {56 vars in 1135 bytes} [Mon Aug 1 13:54:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:30.454152 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32fdcf-90a3-4511-990c-85cb1a9f873c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:30.535208 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:30.556457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:30.576699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 367/732] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:30 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:54:30.579588 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85cf7ef-c030-4dd7-9b9f-caa756181a94 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:30.580188 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85cf7ef-c030-4dd7-9b9f-caa756181a94 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:30.583092 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85cf7ef-c030-4dd7-9b9f-caa756181a94 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:30.586340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 368/733] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:30 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:30.593659 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b7c8b-366e-4eaf-b555-de7008f69e94 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:30.594189 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b7c8b-366e-4eaf-b555-de7008f69e94 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:30.594681 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b7c8b-366e-4eaf-b555-de7008f69e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:30.627947 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32fdcf-90a3-4511-990c-85cb1a9f873c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:30.652869 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-f47b7c8b-366e-4eaf-b555-de7008f69e94 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:30.655027 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-f47b7c8b-366e-4eaf-b555-de7008f69e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:30.677477 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a649a3-4996-4281-86c3-a5ad4d70255b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:30.678123 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a649a3-4996-4281-86c3-a5ad4d70255b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:30.678484 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a649a3-4996-4281-86c3-a5ad4d70255b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:30.678787 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a649a3-4996-4281-86c3-a5ad4d70255b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:30.778109 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a649a3-4996-4281-86c3-a5ad4d70255b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:30.780878 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a649a3-4996-4281-86c3-a5ad4d70255b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:30.804728 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 369/734] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:30 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:54:30.806776 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b7c8b-366e-4eaf-b555-de7008f69e94 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:30.840443 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:30.937986 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 366/735] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:54:30 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 13:54:30.939151 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a649a3-4996-4281-86c3-a5ad4d70255b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:31.038092 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b7c8b-366e-4eaf-b555-de7008f69e94 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:31.110049 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:31.128061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f700db-11d6-4079-a3bb-d10d9ed404ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:31.129208 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f700db-11d6-4079-a3bb-d10d9ed404ae None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:31.129681 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f700db-11d6-4079-a3bb-d10d9ed404ae None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:31.130158 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f700db-11d6-4079-a3bb-d10d9ed404ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:31.148864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a649a3-4996-4281-86c3-a5ad4d70255b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:31.150778 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f700db-11d6-4079-a3bb-d10d9ed404ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:54:31.160761 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-85f700db-11d6-4079-a3bb-d10d9ed404ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:54:31.160761 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:31.160761 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:31.160761 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:31.160761 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:31.160761 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:31.160761 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:31.160761 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:31.160761 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:31.160761 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:31.160761 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:31.160761 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:31.160761 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:54:31.160761 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:54:31.160761 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:54:31.160761 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:54:31.163603 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:54:31.163603 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:54:31.163603 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:54:31.163603 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:54:31.163603 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:31.163603 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:31.163603 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:31.163603 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:31.163603 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:31.163603 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:31.163603 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:31.163603 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:31.163603 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:31.163603 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:31.163603 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:31.163603 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:31.166699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:31.166699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:31.166699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:31.166699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:31.166699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:54:31.166699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:54:31.166699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:54:31.166699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:54:31.166699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:54:31.166699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:54:31.166699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:54:31.166699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:31.166699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 370/736] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 13:54:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:31.166699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f700db-11d6-4079-a3bb-d10d9ed404ae None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:31.224640 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85126fdf-975c-4e88-ba45-8db1f609350f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:31.225761 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85126fdf-975c-4e88-ba45-8db1f609350f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:31.226257 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85126fdf-975c-4e88-ba45-8db1f609350f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:31.226737 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85126fdf-975c-4e88-ba45-8db1f609350f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:31.244208 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:31.252247 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85126fdf-975c-4e88-ba45-8db1f609350f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:54:31.262945 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-85126fdf-975c-4e88-ba45-8db1f609350f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:54:31.262945 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:31.262945 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:31.262945 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:31.262945 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:31.262945 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:31.262945 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:31.262945 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:31.262945 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:31.262945 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:31.262945 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:31.262945 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:31.262945 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:54:31.262945 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:54:31.262945 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:54:31.262945 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:54:31.266126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:54:31.266126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:54:31.266126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:54:31.266126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:54:31.266126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:31.266126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:31.266126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:31.266126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:31.266126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:31.266126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:31.266126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:31.266126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:31.266126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:31.266126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:31.266126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:31.266126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:31.268104 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:31.268104 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:31.268104 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:31.268104 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:31.268104 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:54:31.268104 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:54:31.268104 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:54:31.268104 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:54:31.268104 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:54:31.268104 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:54:31.268104 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:54:31.268104 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:31.268104 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 367/737] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:54:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:31.268104 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29cb76f-e024-4bd2-8ac2-7b059c3139f9 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:31.271198 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29cb76f-e024-4bd2-8ac2-7b059c3139f9 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:31.271198 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29cb76f-e024-4bd2-8ac2-7b059c3139f9 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:31.273437 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 368/738] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:31.274868 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29cb76f-e024-4bd2-8ac2-7b059c3139f9 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:31.392217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f700db-11d6-4079-a3bb-d10d9ed404ae None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:31.459542 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc917ea-e2d3-4146-b62a-6e7ba6108f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:31.460717 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc917ea-e2d3-4146-b62a-6e7ba6108f4c None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:31.461387 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc917ea-e2d3-4146-b62a-6e7ba6108f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:31.461879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc917ea-e2d3-4146-b62a-6e7ba6108f4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:31.483011 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc917ea-e2d3-4146-b62a-6e7ba6108f4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:31.485807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc917ea-e2d3-4146-b62a-6e7ba6108f4c None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:31.489690 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29cb76f-e024-4bd2-8ac2-7b059c3139f9 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:31.520479 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-5cc917ea-e2d3-4146-b62a-6e7ba6108f4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:31.520479 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:31.520479 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:31.520479 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:31.520479 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:31.520479 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:31.520479 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:31.520479 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:31.520479 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:31.520479 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:31.520479 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:31.520479 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:31.520479 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:31.520479 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:31.520479 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:31.522172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:31.522172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:31.522172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:31.522172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:31.522172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:31.522172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:31.522172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:31.522172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:31.522172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:31.522172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:31.522172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:31.522172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:31.522172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:31.522172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:31.522172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:31.522172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:31.522172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:31.523950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:31.523950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:31.523950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:31.523950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:31.523950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 371/739] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:31.524543 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc917ea-e2d3-4146-b62a-6e7ba6108f4c None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:31.573398 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9817f-aaec-44b6-9412-787aaa5483f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:31.574582 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9817f-aaec-44b6-9412-787aaa5483f1 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:31.575152 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9817f-aaec-44b6-9412-787aaa5483f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:31.575723 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9817f-aaec-44b6-9412-787aaa5483f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:31.577215 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f9817f-aaec-44b6-9412-787aaa5483f1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:31.577770 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f9817f-aaec-44b6-9412-787aaa5483f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:31.580475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f9817f-aaec-44b6-9412-787aaa5483f1 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:31.604337 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 369/740] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 13:54:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:31.604337 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0db950-db8f-421b-985c-01de729f3140 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:31.604337 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0db950-db8f-421b-985c-01de729f3140 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:31.604337 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0db950-db8f-421b-985c-01de729f3140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:31.606905 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:31.631869 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-ad0db950-db8f-421b-985c-01de729f3140 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:31.634070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ad0db950-db8f-421b-985c-01de729f3140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:31.726528 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc917ea-e2d3-4146-b62a-6e7ba6108f4c None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:31.737578 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:31.760395 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 370/741] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:31 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:54:31.761886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0db950-db8f-421b-985c-01de729f3140 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:31.791081 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bcdcb1-b7c0-4524-9a93-4eb495e53621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:31.792632 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bcdcb1-b7c0-4524-9a93-4eb495e53621 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:31.793522 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bcdcb1-b7c0-4524-9a93-4eb495e53621 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:31.794276 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bcdcb1-b7c0-4524-9a93-4eb495e53621 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:31.795987 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bcdcb1-b7c0-4524-9a93-4eb495e53621 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:31.797069 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bcdcb1-b7c0-4524-9a93-4eb495e53621 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:31.800084 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bcdcb1-b7c0-4524-9a93-4eb495e53621 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:31.816190 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 372/742] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:54:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:31.818077 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bcdcb1-b7c0-4524-9a93-4eb495e53621 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:31.932868 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0db950-db8f-421b-985c-01de729f3140 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:31.977056 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529b4bb-9b65-4e0b-86bf-ad2d93466cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:31.977786 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529b4bb-9b65-4e0b-86bf-ad2d93466cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:31.978682 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529b4bb-9b65-4e0b-86bf-ad2d93466cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:31.978682 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529b4bb-9b65-4e0b-86bf-ad2d93466cc0 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:31.979068 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529b4bb-9b65-4e0b-86bf-ad2d93466cc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:31.979385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529b4bb-9b65-4e0b-86bf-ad2d93466cc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:31.981118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529b4bb-9b65-4e0b-86bf-ad2d93466cc0 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:32.003625 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 371/743] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:32.003625 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529b4bb-9b65-4e0b-86bf-ad2d93466cc0 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:32.007176 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:32.012449 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bcdcb1-b7c0-4524-9a93-4eb495e53621 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:32.073360 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f44c87-0942-46fc-b4e1-3c2dd525ac5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:32.074373 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f44c87-0942-46fc-b4e1-3c2dd525ac5f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:32.074713 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f44c87-0942-46fc-b4e1-3c2dd525ac5f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:32.075011 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f44c87-0942-46fc-b4e1-3c2dd525ac5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:32.091443 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f44c87-0942-46fc-b4e1-3c2dd525ac5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:32.093022 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f44c87-0942-46fc-b4e1-3c2dd525ac5f None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:32.099117 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:32.115023 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-91f44c87-0942-46fc-b4e1-3c2dd525ac5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:32.115023 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:32.115023 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:32.115023 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:32.115023 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:32.115023 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:32.115023 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:32.115023 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:32.115023 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:32.115023 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:32.115023 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:32.115023 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:32.115023 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:32.115023 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:32.115023 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:32.116884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:32.116884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:32.116884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:32.116884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:32.116884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:32.116884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:32.116884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:32.116884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:32.116884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:32.116884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:32.116884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:32.116884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:32.116884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:32.116884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:32.116884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:32.116884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:32.116884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:32.118560 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:32.118560 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:32.118560 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:32.118560 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:32.118560 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 373/744] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:32.118560 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f44c87-0942-46fc-b4e1-3c2dd525ac5f None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:32.207589 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529b4bb-9b65-4e0b-86bf-ad2d93466cc0 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:32.268486 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d240f3c8-15f7-4d7a-a988-8cb640b38d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:32.269347 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240f3c8-15f7-4d7a-a988-8cb640b38d54 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:32.270109 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240f3c8-15f7-4d7a-a988-8cb640b38d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:32.270547 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240f3c8-15f7-4d7a-a988-8cb640b38d54 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:32.271409 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d240f3c8-15f7-4d7a-a988-8cb640b38d54 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:32.273006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d240f3c8-15f7-4d7a-a988-8cb640b38d54 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:32.292539 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 372/745] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Aug 1 13:54:31 2022] POST /identity/v3/services => generated 240 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:54:32.293838 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d240f3c8-15f7-4d7a-a988-8cb640b38d54 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:32.300305 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f44c87-0942-46fc-b4e1-3c2dd525ac5f None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:32.381971 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc79e9-2290-4ede-baed-086d15c103e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:32.383299 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc79e9-2290-4ede-baed-086d15c103e1 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:32.383721 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc79e9-2290-4ede-baed-086d15c103e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:32.384169 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc79e9-2290-4ede-baed-086d15c103e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:32.402699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cc79e9-2290-4ede-baed-086d15c103e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:32.405305 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cc79e9-2290-4ede-baed-086d15c103e1 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:32.421810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:32.429649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-62cc79e9-2290-4ede-baed-086d15c103e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:32.429649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:32.429649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:32.429649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:32.429649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:32.429649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:32.429649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:32.429649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:32.429649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:32.429649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:32.429649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:32.429649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:32.429649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:32.429649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:32.429649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:32.431745 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:32.431745 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:32.431745 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:32.431745 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:32.431745 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:32.431745 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:32.431745 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:32.431745 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:32.431745 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:32.431745 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:32.431745 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:32.431745 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:32.431745 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:32.431745 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:32.431745 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:32.431745 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:32.431745 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:32.433753 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:32.433753 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:32.433753 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:32.433753 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:32.436621 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 374/746] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:32.437696 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc79e9-2290-4ede-baed-086d15c103e1 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:32.485753 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d240f3c8-15f7-4d7a-a988-8cb640b38d54 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:32.500887 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:32.543772 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6aa67-7eba-481f-a512-6651764d9cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:32.545835 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6aa67-7eba-481f-a512-6651764d9cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:32.546622 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6aa67-7eba-481f-a512-6651764d9cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:32.547768 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6aa67-7eba-481f-a512-6651764d9cf3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:32.558780 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6aa67-7eba-481f-a512-6651764d9cf3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:32.561000 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6aa67-7eba-481f-a512-6651764d9cf3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:32.579886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-faf6aa67-7eba-481f-a512-6651764d9cf3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:32.579886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:32.579886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:32.579886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:32.579886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:32.579886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:32.579886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:32.579886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:32.579886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:32.579886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:32.579886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:32.579886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:32.579886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:54:32.579886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:54:32.579886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:54:32.582006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:54:32.582006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:32.582006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:32.582006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:32.582006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:32.582006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:32.582006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:32.582006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:32.582006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:32.582006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:32.582006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:32.582006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:32.582006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:32.582006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:32.582006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:32.582006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:32.582006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:32.584074 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:54:32.584074 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:54:32.584074 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:54:32.584074 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:54:32.584074 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:54:32.584074 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:54:32.584074 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:32.584074 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:32.586055 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 373/747] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:54:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:32.586369 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6fabe-6c25-4f23-9896-831828f4edf7 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:32.586755 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6fabe-6c25-4f23-9896-831828f4edf7 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:32.587127 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6fabe-6c25-4f23-9896-831828f4edf7 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:32.589610 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 374/748] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:32.590829 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a6fabe-6c25-4f23-9896-831828f4edf7 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:32.594740 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc79e9-2290-4ede-baed-086d15c103e1 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:32.656582 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47461ed0-bb97-4a56-aeb5-3e60629560d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:32.657806 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47461ed0-bb97-4a56-aeb5-3e60629560d7 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:32.658300 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47461ed0-bb97-4a56-aeb5-3e60629560d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:32.658803 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47461ed0-bb97-4a56-aeb5-3e60629560d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:32.660126 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47461ed0-bb97-4a56-aeb5-3e60629560d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:32.660699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47461ed0-bb97-4a56-aeb5-3e60629560d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:32.663353 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47461ed0-bb97-4a56-aeb5-3e60629560d7 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:32.692073 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 375/749] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:32.692073 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47461ed0-bb97-4a56-aeb5-3e60629560d7 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:32.755349 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a6fabe-6c25-4f23-9896-831828f4edf7 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:32.799254 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a0563-0407-4d73-8ee1-b6b1f2ad7bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:32.799996 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a0563-0407-4d73-8ee1-b6b1f2ad7bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:32.800387 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a0563-0407-4d73-8ee1-b6b1f2ad7bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:32.800731 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a0563-0407-4d73-8ee1-b6b1f2ad7bb8 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:32.801533 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293a0563-0407-4d73-8ee1-b6b1f2ad7bb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:32.801897 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293a0563-0407-4d73-8ee1-b6b1f2ad7bb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:32.803559 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293a0563-0407-4d73-8ee1-b6b1f2ad7bb8 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:32.813504 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:32.819734 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 375/750] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:32.822107 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fc2da-49a2-426d-a5a9-4ea279c65e03 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:32.822595 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fc2da-49a2-426d-a5a9-4ea279c65e03 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:32.823078 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fc2da-49a2-426d-a5a9-4ea279c65e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:32.846940 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-eb5fc2da-49a2-426d-a5a9-4ea279c65e03 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:32.848316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-eb5fc2da-49a2-426d-a5a9-4ea279c65e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:32.879005 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:32.910672 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47461ed0-bb97-4a56-aeb5-3e60629560d7 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:32.968330 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 376/751] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:54:32.968330 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5fc2da-49a2-426d-a5a9-4ea279c65e03 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:32.972207 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584fda2c-e15f-4b9a-8f4e-f6b82529888e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:32.973463 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584fda2c-e15f-4b9a-8f4e-f6b82529888e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:32.974071 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584fda2c-e15f-4b9a-8f4e-f6b82529888e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:32.974711 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584fda2c-e15f-4b9a-8f4e-f6b82529888e None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:32.976193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584fda2c-e15f-4b9a-8f4e-f6b82529888e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:32.976867 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584fda2c-e15f-4b9a-8f4e-f6b82529888e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:32.979700 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584fda2c-e15f-4b9a-8f4e-f6b82529888e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:33.008785 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 376/752] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:54:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:33.010382 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584fda2c-e15f-4b9a-8f4e-f6b82529888e None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:33.149689 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5fc2da-49a2-426d-a5a9-4ea279c65e03 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:33.213190 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d33af-ba02-41a3-947f-2320eb6a5a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:33.214224 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d33af-ba02-41a3-947f-2320eb6a5a4f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:33.214672 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d33af-ba02-41a3-947f-2320eb6a5a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:33.215121 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d33af-ba02-41a3-947f-2320eb6a5a4f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:33.228690 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21d33af-ba02-41a3-947f-2320eb6a5a4f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:33.231178 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21d33af-ba02-41a3-947f-2320eb6a5a4f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:33.252460 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-e21d33af-ba02-41a3-947f-2320eb6a5a4f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 13:54:33.252460 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:33.252460 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:33.252460 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:33.252460 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:33.252460 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:33.252460 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:33.252460 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:33.252460 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:33.252460 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:33.252460 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:33.252460 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:33.252460 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:54:33.252460 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:54:33.252460 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:54:33.252460 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:54:33.254388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:33.254388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:33.254388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:54:33.254388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:33.254388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:54:33.254388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:33.254388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:33.254388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:33.254388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:33.254388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:33.254388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:33.254388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:33.254388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:33.254388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:33.254388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:33.254388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:33.254388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:33.254388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:33.264256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:33.264256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:33.264256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:54:33.264256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:54:33.264256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:54:33.264256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:54:33.264256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:54:33.264256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:54:33.264256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 13:54:33.264256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:33.264256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 377/753] 10.222.0.45 () {56 vars in 1117 bytes} [Mon Aug 1 13:54:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:33.264256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d33af-ba02-41a3-947f-2320eb6a5a4f None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:33.308727 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:33.386279 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584fda2c-e15f-4b9a-8f4e-f6b82529888e None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:33.427021 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:33.463323 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f44f2a2-b10b-4fb9-b13a-c7d9014ac8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:33.464489 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44f2a2-b10b-4fb9-b13a-c7d9014ac8aa None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:33.464981 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44f2a2-b10b-4fb9-b13a-c7d9014ac8aa None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:33.465458 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44f2a2-b10b-4fb9-b13a-c7d9014ac8aa None admin] PATH_INFO: `/v3/users/nova` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:33.467562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d33af-ba02-41a3-947f-2320eb6a5a4f None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:33.480488 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f44f2a2-b10b-4fb9-b13a-c7d9014ac8aa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:33.483084 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f44f2a2-b10b-4fb9-b13a-c7d9014ac8aa None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:33.505882 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-6f44f2a2-b10b-4fb9-b13a-c7d9014ac8aa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:54:33.505882 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:33.505882 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:33.505882 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:33.505882 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:33.505882 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:33.505882 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:33.505882 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:33.505882 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:33.505882 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:33.505882 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:33.505882 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:33.505882 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:54:33.505882 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:54:33.505882 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:54:33.505882 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:54:33.507364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:33.507364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:33.507364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:54:33.507364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:33.507364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:54:33.507364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:33.507364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:33.507364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:33.507364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:33.507364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:33.507364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:33.507364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:33.507364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:33.507364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:33.507364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:33.507364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:33.507364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:33.507364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:33.508365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:33.508365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:33.508365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:54:33.508365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:54:33.508365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:54:33.508365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:54:33.508365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:54:33.508365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:54:33.508365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:54:33.508365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:33.508365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 377/754] 10.222.0.45 () {56 vars in 1093 bytes} [Mon Aug 1 13:54:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 499 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:33.510128 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f44f2a2-b10b-4fb9-b13a-c7d9014ac8aa None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:33.561237 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ba2ee-eaad-440e-86e1-cb4ed670f363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:33.562327 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ba2ee-eaad-440e-86e1-cb4ed670f363 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:33.562643 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ba2ee-eaad-440e-86e1-cb4ed670f363 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:33.563084 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ba2ee-eaad-440e-86e1-cb4ed670f363 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:33.564134 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ba2ee-eaad-440e-86e1-cb4ed670f363 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:33.566658 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ba2ee-eaad-440e-86e1-cb4ed670f363 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:33.593511 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-184ba2ee-eaad-440e-86e1-cb4ed670f363 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 13:54:33.593511 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:33.593511 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:54:33.593511 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:54:33.593511 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:54:33.593511 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:54:33.593511 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:54:33.593511 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:54:33.593511 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 13:54:33.593511 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:33.593511 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:54:33.593511 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:33.593511 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:33.593511 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:33.593511 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:33.593511 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:33.602881 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:33.602881 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:33.602881 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:33.602881 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:33.602881 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:33.602881 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:33.602881 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:33.602881 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:54:33.602881 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:54:33.602881 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:54:33.602881 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:54:33.602881 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:33.602881 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:33.602881 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:33.602881 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:33.602881 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:33.604882 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:33.604882 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:33.604882 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:33.604882 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:33.604882 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:33.604882 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:33.604882 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:33.604882 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:33.604882 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:33.604882 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:33.604882 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:33.604882 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:54:33.604882 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:54:33.604882 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 13:54:33.604882 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:33.604882 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 378/755] 10.222.0.45 () {56 vars in 1129 bytes} [Mon Aug 1 13:54:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:33.606419 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ba2ee-eaad-440e-86e1-cb4ed670f363 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:33.705310 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f44f2a2-b10b-4fb9-b13a-c7d9014ac8aa None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:33.766856 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:33.768650 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabab5f2-042e-42a1-9cc0-1fceb12f1cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:33.769933 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabab5f2-042e-42a1-9cc0-1fceb12f1cca None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:33.770328 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabab5f2-042e-42a1-9cc0-1fceb12f1cca None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:33.770701 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabab5f2-042e-42a1-9cc0-1fceb12f1cca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:33.771624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabab5f2-042e-42a1-9cc0-1fceb12f1cca None admin] RBAC: Adding query filter params (role.id=3e1a9ff016c840c5b51ef7e3da9900ac, scope.project.id=36f063d8f65d494a89dbbd72c53417c8, user.id=0cda7f365ba0405daaf86870960f12d2) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:33.772031 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabab5f2-042e-42a1-9cc0-1fceb12f1cca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:33.774226 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabab5f2-042e-42a1-9cc0-1fceb12f1cca None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:33.792329 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 378/756] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:54:33 2022] GET /identity/v3/role_assignments?user.id=0cda7f365ba0405daaf86870960f12d2&scope.project.id=36f063d8f65d494a89dbbd72c53417c8&role.id=3e1a9ff016c840c5b51ef7e3da9900ac => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:33.792329 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabab5f2-042e-42a1-9cc0-1fceb12f1cca None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:33.802037 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ba2ee-eaad-440e-86e1-cb4ed670f363 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:33.870755 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34efbbdc-29fc-4d8b-90e1-2d354cd006b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:33.871900 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34efbbdc-29fc-4d8b-90e1-2d354cd006b1 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:33.872415 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34efbbdc-29fc-4d8b-90e1-2d354cd006b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:33.872890 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34efbbdc-29fc-4d8b-90e1-2d354cd006b1 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:33.874250 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34efbbdc-29fc-4d8b-90e1-2d354cd006b1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:33.874827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34efbbdc-29fc-4d8b-90e1-2d354cd006b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:33.877400 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34efbbdc-29fc-4d8b-90e1-2d354cd006b1 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:33.904826 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:33.918591 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 379/757] 10.222.0.45 () {56 vars in 1149 bytes} [Mon Aug 1 13:54:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:33.920175 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34efbbdc-29fc-4d8b-90e1-2d354cd006b1 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:34.017181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabab5f2-042e-42a1-9cc0-1fceb12f1cca None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:34.082074 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb844e2-4c40-4f3d-bc9d-9aba82c84aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:34.082722 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb844e2-4c40-4f3d-bc9d-9aba82c84aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:34.083070 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb844e2-4c40-4f3d-bc9d-9aba82c84aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:34.083508 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb844e2-4c40-4f3d-bc9d-9aba82c84aa6 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:34.085035 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb844e2-4c40-4f3d-bc9d-9aba82c84aa6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:34.085543 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb844e2-4c40-4f3d-bc9d-9aba82c84aa6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:34.089635 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb844e2-4c40-4f3d-bc9d-9aba82c84aa6 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:34.124415 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 379/758] 10.222.0.45 () {56 vars in 1133 bytes} [Mon Aug 1 13:54:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:34.126547 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb844e2-4c40-4f3d-bc9d-9aba82c84aa6 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:34.138149 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34efbbdc-29fc-4d8b-90e1-2d354cd006b1 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:34.189197 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddfc2a5-8a18-4be1-9191-09473f6332eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:34.189846 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddfc2a5-8a18-4be1-9191-09473f6332eb None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:34.190123 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddfc2a5-8a18-4be1-9191-09473f6332eb None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:34.190539 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddfc2a5-8a18-4be1-9191-09473f6332eb None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:34.191765 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddfc2a5-8a18-4be1-9191-09473f6332eb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:34.192518 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddfc2a5-8a18-4be1-9191-09473f6332eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:34.195226 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddfc2a5-8a18-4be1-9191-09473f6332eb None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:34.203092 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 380/759] 10.222.0.45 () {56 vars in 1124 bytes} [Mon Aug 1 13:54:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:34.203831 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddfc2a5-8a18-4be1-9191-09473f6332eb None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:34.216882 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:34.334377 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb844e2-4c40-4f3d-bc9d-9aba82c84aa6 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:34.337817 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:34.391296 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddfc2a5-8a18-4be1-9191-09473f6332eb None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:34.401387 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699a86a-eb66-4563-b5b2-831df01949c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:34.403436 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699a86a-eb66-4563-b5b2-831df01949c1 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:34.404235 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699a86a-eb66-4563-b5b2-831df01949c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:34.405091 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699a86a-eb66-4563-b5b2-831df01949c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:34.462936 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389cfb7c-0283-4f27-b532-97327b5d7e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:34.464350 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389cfb7c-0283-4f27-b532-97327b5d7e8e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:34.464818 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389cfb7c-0283-4f27-b532-97327b5d7e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:34.465263 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389cfb7c-0283-4f27-b532-97327b5d7e8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:34.487378 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389cfb7c-0283-4f27-b532-97327b5d7e8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:34.490286 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389cfb7c-0283-4f27-b532-97327b5d7e8e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:34.520303 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-389cfb7c-0283-4f27-b532-97327b5d7e8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:34.520303 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:34.520303 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:34.520303 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:34.520303 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:34.520303 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:34.520303 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:34.520303 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:34.520303 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:34.520303 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:34.520303 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:34.520303 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:34.520303 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:34.520303 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:34.520303 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:34.521783 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:34.521783 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:34.521783 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:34.521783 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:34.521783 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:34.521783 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:34.521783 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:34.521783 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:34.521783 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:34.521783 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:34.521783 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:34.521783 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:34.521783 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:34.521783 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:34.521783 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:34.521783 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:34.521783 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:34.523385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:34.523385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:34.523385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:34.523385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:34.528493 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 381/760] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:34.530158 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389cfb7c-0283-4f27-b532-97327b5d7e8e None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:34.538660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699a86a-eb66-4563-b5b2-831df01949c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:34.540943 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699a86a-eb66-4563-b5b2-831df01949c1 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:34.657360 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 380/761] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:54:34 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:54:34.657559 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:34.659216 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699a86a-eb66-4563-b5b2-831df01949c1 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:34.663650 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:34.743268 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389cfb7c-0283-4f27-b532-97327b5d7e8e None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:34.771166 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699a86a-eb66-4563-b5b2-831df01949c1 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:34.809801 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3b9bf-1c9b-4768-b6f8-92442c7db640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:34.810847 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3b9bf-1c9b-4768-b6f8-92442c7db640 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:34.811328 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3b9bf-1c9b-4768-b6f8-92442c7db640 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:34.811835 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c93c8-1b84-4165-9f8b-d37e5b939355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:34.811961 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3b9bf-1c9b-4768-b6f8-92442c7db640 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:34.812979 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac3b9bf-1c9b-4768-b6f8-92442c7db640 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:34.813088 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c93c8-1b84-4165-9f8b-d37e5b939355 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:34.813508 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c93c8-1b84-4165-9f8b-d37e5b939355 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:34.813792 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac3b9bf-1c9b-4768-b6f8-92442c7db640 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:34.814005 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c93c8-1b84-4165-9f8b-d37e5b939355 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:34.815227 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c93c8-1b84-4165-9f8b-d37e5b939355 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:34.815788 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c93c8-1b84-4165-9f8b-d37e5b939355 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:34.816149 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac3b9bf-1c9b-4768-b6f8-92442c7db640 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:34.818470 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c93c8-1b84-4165-9f8b-d37e5b939355 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:34.828703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 382/762] 10.222.0.45 () {56 vars in 1124 bytes} [Mon Aug 1 13:54:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:34.830352 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 381/763] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:34.840325 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c93c8-1b84-4165-9f8b-d37e5b939355 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:34.840528 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3b9bf-1c9b-4768-b6f8-92442c7db640 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:34.972429 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:35.030161 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c93c8-1b84-4165-9f8b-d37e5b939355 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:35.069478 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3b9bf-1c9b-4768-b6f8-92442c7db640 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:35.081437 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:35.105272 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0109ee-3681-43bb-acf8-e8c657ded43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:35.106383 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0109ee-3681-43bb-acf8-e8c657ded43e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:35.106872 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0109ee-3681-43bb-acf8-e8c657ded43e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:35.107360 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0109ee-3681-43bb-acf8-e8c657ded43e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:35.124746 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0109ee-3681-43bb-acf8-e8c657ded43e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:35.127459 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0109ee-3681-43bb-acf8-e8c657ded43e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:35.148634 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b586ae-33ab-4e84-bbcd-d289da887ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:35.149694 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b586ae-33ab-4e84-bbcd-d289da887ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:35.150156 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b586ae-33ab-4e84-bbcd-d289da887ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:35.150846 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b586ae-33ab-4e84-bbcd-d289da887ee5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:35.152567 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b586ae-33ab-4e84-bbcd-d289da887ee5 None admin] RBAC: Adding query filter params (interface=public, service_id=1f4398179c01440cb0e0ea94b57e1dc2, region_id=RegionOne) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:35.153373 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-5b0109ee-3681-43bb-acf8-e8c657ded43e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:35.153373 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:35.153373 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:35.153373 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:35.153373 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:35.153373 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:35.153373 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:35.153373 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:35.153373 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:35.153373 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:35.153373 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:35.153373 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:35.153373 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:35.153373 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:35.153373 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:35.154965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:35.154965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:35.154965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:35.154965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:35.154965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:35.154965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:35.154965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:35.154965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:35.154965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:35.154965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:35.154965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:35.154965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:35.154965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:35.154965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:35.154965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:35.154965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:35.154965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:35.156727 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:35.156727 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:35.156727 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:35.156727 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:35.156727 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 383/764] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:35.157241 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0109ee-3681-43bb-acf8-e8c657ded43e None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:35.157810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b586ae-33ab-4e84-bbcd-d289da887ee5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:35.160874 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b586ae-33ab-4e84-bbcd-d289da887ee5 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:35.178802 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 382/765] 10.222.0.45 () {56 vars in 1251 bytes} [Mon Aug 1 13:54:34 2022] GET /identity/v3/endpoints?service_id=1f4398179c01440cb0e0ea94b57e1dc2&interface=public®ion_id=RegionOne => generated 194 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:35.180762 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc3322-7315-434d-8860-8e89f02607f6 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:35.181217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc3322-7315-434d-8860-8e89f02607f6 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:35.181637 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc3322-7315-434d-8860-8e89f02607f6 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:35.183202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 383/766] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:35.184161 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc3322-7315-434d-8860-8e89f02607f6 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:35.365267 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0109ee-3681-43bb-acf8-e8c657ded43e None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:35.377989 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc3322-7315-434d-8860-8e89f02607f6 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:35.427863 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:35.428643 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c8676b-730b-4538-ac2e-53be9c9fe02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:35.429193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8676b-730b-4538-ac2e-53be9c9fe02c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:35.429479 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8676b-730b-4538-ac2e-53be9c9fe02c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:35.429791 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8676b-730b-4538-ac2e-53be9c9fe02c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:35.439763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c8676b-730b-4538-ac2e-53be9c9fe02c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:35.442726 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c8676b-730b-4538-ac2e-53be9c9fe02c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:35.444866 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f531ff-3715-4467-8c73-44da23e9ae54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:35.445812 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f531ff-3715-4467-8c73-44da23e9ae54 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:35.446125 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f531ff-3715-4467-8c73-44da23e9ae54 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:35.446602 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f531ff-3715-4467-8c73-44da23e9ae54 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:35.447962 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f531ff-3715-4467-8c73-44da23e9ae54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:35.448554 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f531ff-3715-4467-8c73-44da23e9ae54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:35.451204 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f531ff-3715-4467-8c73-44da23e9ae54 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:35.463546 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-12c8676b-730b-4538-ac2e-53be9c9fe02c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:35.463546 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:35.463546 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:35.463546 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:35.463546 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:35.463546 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:35.463546 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:35.463546 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:35.463546 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:35.463546 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:35.463546 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:35.463546 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:35.463546 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:54:35.463546 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:54:35.463546 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:54:35.465376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:54:35.465376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:35.465376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:35.465376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:35.465376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:35.465376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:35.465376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:35.465376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:35.465376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:35.465376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:35.465376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:35.465376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:35.465376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:35.465376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:35.465376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:35.465376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:35.465376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:35.467007 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:54:35.467007 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:54:35.467007 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:54:35.467007 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:54:35.467007 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:54:35.467007 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:54:35.467007 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:35.467007 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:35.467007 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 384/767] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:54:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:35.467007 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c8676b-730b-4538-ac2e-53be9c9fe02c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:35.483794 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 384/768] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:35.486560 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c792c-1dad-4b09-8aac-2630a1cf538d None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:35.487437 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c792c-1dad-4b09-8aac-2630a1cf538d None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:35.488157 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c792c-1dad-4b09-8aac-2630a1cf538d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:35.522290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-cb8c792c-1dad-4b09-8aac-2630a1cf538d None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:35.525141 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-cb8c792c-1dad-4b09-8aac-2630a1cf538d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:35.540431 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:35.657597 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 385/769] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:54:35.659049 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c792c-1dad-4b09-8aac-2630a1cf538d None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:35.680947 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c8676b-730b-4538-ac2e-53be9c9fe02c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:35.747390 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c3231d-3add-47c6-a8f1-2499c551d15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:35.748478 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c3231d-3add-47c6-a8f1-2499c551d15e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:35.748948 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c3231d-3add-47c6-a8f1-2499c551d15e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:35.749417 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c3231d-3add-47c6-a8f1-2499c551d15e None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:35.750524 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c3231d-3add-47c6-a8f1-2499c551d15e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:35.752122 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c3231d-3add-47c6-a8f1-2499c551d15e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:35.768708 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 385/770] 10.222.0.45 () {56 vars in 1087 bytes} [Mon Aug 1 13:54:35 2022] GET /identity/v3/services => generated 772 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:35.768708 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c3231d-3add-47c6-a8f1-2499c551d15e None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:35.883184 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c792c-1dad-4b09-8aac-2630a1cf538d None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:35.923485 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:35.962272 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7733c3ec-2520-4007-9045-7063659f7311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:35.964102 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733c3ec-2520-4007-9045-7063659f7311 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:35.964854 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733c3ec-2520-4007-9045-7063659f7311 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:35.965543 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733c3ec-2520-4007-9045-7063659f7311 None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:35.967482 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7733c3ec-2520-4007-9045-7063659f7311 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:35.968426 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7733c3ec-2520-4007-9045-7063659f7311 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:35.972264 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7733c3ec-2520-4007-9045-7063659f7311 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:35.980394 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:35.989174 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c3231d-3add-47c6-a8f1-2499c551d15e None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:36.001504 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 386/771] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:54:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:36.003108 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7733c3ec-2520-4007-9045-7063659f7311 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:36.074851 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057537c-eedf-4717-a3c8-8a1e72517227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:36.076211 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057537c-eedf-4717-a3c8-8a1e72517227 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:36.076904 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057537c-eedf-4717-a3c8-8a1e72517227 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:36.077503 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057537c-eedf-4717-a3c8-8a1e72517227 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:36.096117 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9057537c-eedf-4717-a3c8-8a1e72517227 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:36.099962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9057537c-eedf-4717-a3c8-8a1e72517227 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:36.130645 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-9057537c-eedf-4717-a3c8-8a1e72517227 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:36.130645 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:36.130645 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:36.130645 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:36.130645 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:36.130645 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:36.130645 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:36.130645 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:36.130645 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:36.130645 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:36.130645 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:36.130645 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:36.130645 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:54:36.130645 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:54:36.130645 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:54:36.132078 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:54:36.132078 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:36.132078 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:36.132078 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:36.132078 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:36.132078 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:36.132078 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:36.132078 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:36.132078 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:36.132078 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:36.132078 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:36.132078 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:36.132078 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:36.132078 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:36.132078 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:36.132078 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:36.132078 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:36.133639 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:54:36.133639 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:54:36.133639 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:54:36.133639 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:54:36.133639 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:54:36.133639 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:54:36.133639 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:36.133639 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:36.134982 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 386/772] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:54:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:36.140436 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057537c-eedf-4717-a3c8-8a1e72517227 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:36.224181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7733c3ec-2520-4007-9045-7063659f7311 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:36.295343 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d4ad4-671a-4349-924c-52cea4e7b8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:36.296320 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d4ad4-671a-4349-924c-52cea4e7b8fd None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:36.296732 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d4ad4-671a-4349-924c-52cea4e7b8fd None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:36.297142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d4ad4-671a-4349-924c-52cea4e7b8fd None admin] PATH_INFO: `/v3/services/network` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:36.298423 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d4ad4-671a-4349-924c-52cea4e7b8fd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:36.301935 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d4ad4-671a-4349-924c-52cea4e7b8fd None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:36.321846 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-588d4ad4-671a-4349-924c-52cea4e7b8fd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 13:54:36.321846 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:36.321846 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:54:36.321846 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:54:36.321846 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:54:36.321846 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:54:36.321846 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:54:36.321846 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:54:36.321846 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 13:54:36.321846 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:36.321846 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:54:36.321846 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:36.321846 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:36.321846 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:36.321846 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:36.321846 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:36.321846 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:36.324648 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:36.324648 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:36.324648 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:36.324648 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:36.324648 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:36.324648 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:36.324648 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:54:36.324648 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:54:36.324648 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:54:36.324648 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:54:36.324648 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:36.324648 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:36.324648 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:36.324648 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:36.324648 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:36.324648 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:36.327163 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:36.327163 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:36.327163 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:36.327163 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:36.327163 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:36.327163 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:36.327163 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:36.327163 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:36.327163 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:36.327163 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:36.327163 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:54:36.327163 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:54:36.327163 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 13:54:36.327163 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:36.327163 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 387/773] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:54:36 2022] GET /identity/v3/services/network => generated 88 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:36.337774 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d4ad4-671a-4349-924c-52cea4e7b8fd None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:36.383580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057537c-eedf-4717-a3c8-8a1e72517227 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:36.417499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:36.455198 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36ce9b-f0d1-4bbc-b3b3-69133b56154b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:36.456516 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36ce9b-f0d1-4bbc-b3b3-69133b56154b None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:36.456978 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36ce9b-f0d1-4bbc-b3b3-69133b56154b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:36.457403 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36ce9b-f0d1-4bbc-b3b3-69133b56154b None admin] PATH_INFO: `/v3/projects/36f063d8f65d494a89dbbd72c53417c8/users/5b3adc14dcaf4ac79b43bb97fa1ba1d4/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:36.460865 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef36ce9b-f0d1-4bbc-b3b3-69133b56154b None admin] RBAC: Authorizing `identity:create_grant(project_id=36f063d8f65d494a89dbbd72c53417c8, user_id=5b3adc14dcaf4ac79b43bb97fa1ba1d4, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:36.463448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef36ce9b-f0d1-4bbc-b3b3-69133b56154b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:36.483705 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 387/774] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 13:54:36 2022] PUT /identity/v3/projects/36f063d8f65d494a89dbbd72c53417c8/users/5b3adc14dcaf4ac79b43bb97fa1ba1d4/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:54:36.485040 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36ce9b-f0d1-4bbc-b3b3-69133b56154b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:36.513503 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:36.594806 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d4ad4-671a-4349-924c-52cea4e7b8fd None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:36.636243 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36ce9b-f0d1-4bbc-b3b3-69133b56154b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:36.683252 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b7813-8b87-49dd-b209-16848c2e74a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:36.684218 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b7813-8b87-49dd-b209-16848c2e74a2 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:36.684492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b7813-8b87-49dd-b209-16848c2e74a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:36.684993 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b7813-8b87-49dd-b209-16848c2e74a2 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:36.686315 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b7813-8b87-49dd-b209-16848c2e74a2 None admin] RBAC: Adding query filter params (name=network) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:36.686622 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b7813-8b87-49dd-b209-16848c2e74a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:36.688118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b7813-8b87-49dd-b209-16848c2e74a2 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:36.694252 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 388/775] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Aug 1 13:54:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:36.700592 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b7813-8b87-49dd-b209-16848c2e74a2 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:36.718156 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f7eb87-b944-4657-809b-85391fca0fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:36.718986 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f7eb87-b944-4657-809b-85391fca0fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:36.719405 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f7eb87-b944-4657-809b-85391fca0fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:36.719834 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f7eb87-b944-4657-809b-85391fca0fc6 None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:36.720753 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f7eb87-b944-4657-809b-85391fca0fc6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:36.721066 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f7eb87-b944-4657-809b-85391fca0fc6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:36.723284 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f7eb87-b944-4657-809b-85391fca0fc6 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:36.768557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 388/776] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:54:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 442 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:36.779735 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f7eb87-b944-4657-809b-85391fca0fc6 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:36.829676 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:36.878543 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b7813-8b87-49dd-b209-16848c2e74a2 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:36.945442 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ff12b-8e75-4d56-8a8a-316a3e94394f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:36.946085 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ff12b-8e75-4d56-8a8a-316a3e94394f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:36.946417 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ff12b-8e75-4d56-8a8a-316a3e94394f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:36.947187 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ff12b-8e75-4d56-8a8a-316a3e94394f None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:36.948678 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ff12b-8e75-4d56-8a8a-316a3e94394f None admin] RBAC: Adding query filter params (type=network) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:36.948980 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ff12b-8e75-4d56-8a8a-316a3e94394f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:36.951373 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ff12b-8e75-4d56-8a8a-316a3e94394f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:36.959313 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 389/777] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Aug 1 13:54:36 2022] GET /identity/v3/services?type=network => generated 350 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:36.966188 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ff12b-8e75-4d56-8a8a-316a3e94394f None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:37.014794 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:37.022762 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f7eb87-b944-4657-809b-85391fca0fc6 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:37.105368 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04781e5b-a240-4aba-8b62-d75a68cb6610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:37.106301 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04781e5b-a240-4aba-8b62-d75a68cb6610 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:37.106717 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04781e5b-a240-4aba-8b62-d75a68cb6610 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:37.107138 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04781e5b-a240-4aba-8b62-d75a68cb6610 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:37.108447 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04781e5b-a240-4aba-8b62-d75a68cb6610 None admin] RBAC: Adding query filter params (role.id=d17972f7fe97462b8f9f08aed9bc9cb5, scope.project.id=36f063d8f65d494a89dbbd72c53417c8, user.id=e07e83c0bdba47d39c09308eba1bdaae) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:37.109000 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04781e5b-a240-4aba-8b62-d75a68cb6610 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:37.111517 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04781e5b-a240-4aba-8b62-d75a68cb6610 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:37.131516 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 389/778] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:54:36 2022] GET /identity/v3/role_assignments?user.id=e07e83c0bdba47d39c09308eba1bdaae&scope.project.id=36f063d8f65d494a89dbbd72c53417c8&role.id=d17972f7fe97462b8f9f08aed9bc9cb5 => generated 259 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:37.159482 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ff12b-8e75-4d56-8a8a-316a3e94394f None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:37.191166 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722cdc9-bc02-4b86-83b6-4d93b25d49e8 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:37.191903 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722cdc9-bc02-4b86-83b6-4d93b25d49e8 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:37.192437 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722cdc9-bc02-4b86-83b6-4d93b25d49e8 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:37.194437 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 390/779] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:54:37.200829 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e2842-6f30-4cf3-94d8-4aed88bd1ea6 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:37.201433 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e2842-6f30-4cf3-94d8-4aed88bd1ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:37.202007 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e2842-6f30-4cf3-94d8-4aed88bd1ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:37.212678 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fbe8a3-24b7-4972-a832-4f7dd3408f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:37.213801 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fbe8a3-24b7-4972-a832-4f7dd3408f40 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:37.214294 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fbe8a3-24b7-4972-a832-4f7dd3408f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:37.214760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fbe8a3-24b7-4972-a832-4f7dd3408f40 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:37.216595 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fbe8a3-24b7-4972-a832-4f7dd3408f40 None admin] RBAC: Adding query filter params (interface=public, service_id=c5d5f239eb5e4ae597cbb8db8151e3ed, region_id=RegionOne) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:37.217113 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fbe8a3-24b7-4972-a832-4f7dd3408f40 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:37.221019 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fbe8a3-24b7-4972-a832-4f7dd3408f40 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:37.226682 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-c10e2842-6f30-4cf3-94d8-4aed88bd1ea6 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:37.228412 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c10e2842-6f30-4cf3-94d8-4aed88bd1ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:37.244314 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 390/780] 10.222.0.45 () {56 vars in 1251 bytes} [Mon Aug 1 13:54:36 2022] GET /identity/v3/endpoints?service_id=c5d5f239eb5e4ae597cbb8db8151e3ed&interface=public®ion_id=RegionOne => generated 194 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:54:37.248160 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fbe8a3-24b7-4972-a832-4f7dd3408f40 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:37.279270 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:37.461932 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 391/781] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:54:37.503183 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e2842-6f30-4cf3-94d8-4aed88bd1ea6 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:37.535594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fbe8a3-24b7-4972-a832-4f7dd3408f40 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:37.615484 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23954dd9-f677-417e-ae5c-0f620dc3b5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:37.617308 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23954dd9-f677-417e-ae5c-0f620dc3b5df None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:37.618101 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23954dd9-f677-417e-ae5c-0f620dc3b5df None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:37.618825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23954dd9-f677-417e-ae5c-0f620dc3b5df None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:37.620974 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23954dd9-f677-417e-ae5c-0f620dc3b5df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:37.624023 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23954dd9-f677-417e-ae5c-0f620dc3b5df None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:37.640481 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 391/782] 10.222.0.45 () {56 vars in 1087 bytes} [Mon Aug 1 13:54:37 2022] GET /identity/v3/services => generated 772 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:37.750859 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:37.762424 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e2842-6f30-4cf3-94d8-4aed88bd1ea6 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:37.853317 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b2a75-5053-4d6f-a70d-ab7010b06850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:37.853922 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b2a75-5053-4d6f-a70d-ab7010b06850 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:37.854197 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b2a75-5053-4d6f-a70d-ab7010b06850 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:37.854466 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b2a75-5053-4d6f-a70d-ab7010b06850 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:37.875294 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590b2a75-5053-4d6f-a70d-ab7010b06850 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:54:37.884875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-590b2a75-5053-4d6f-a70d-ab7010b06850 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:54:37.884875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:37.884875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:37.884875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:37.884875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:37.884875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:37.884875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:37.884875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:37.884875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:37.884875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:37.884875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:37.884875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:37.884875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:54:37.884875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:54:37.884875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:54:37.884875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:54:37.886594 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:54:37.886594 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:54:37.886594 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:54:37.886594 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:54:37.886594 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:37.886594 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:37.886594 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:37.886594 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:37.886594 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:37.886594 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:37.886594 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:37.886594 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:37.886594 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:37.886594 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:37.886594 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:37.886594 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:37.887468 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:37.887468 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:37.887468 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:37.887468 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:37.887468 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:54:37.887468 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:54:37.887468 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:54:37.887468 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:54:37.887468 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:54:37.887468 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:54:37.887468 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:54:37.887468 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:37.887468 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 392/783] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:54:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:54:37.899406 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23954dd9-f677-417e-ae5c-0f620dc3b5df None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:38.087921 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:38.141199 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23954dd9-f677-417e-ae5c-0f620dc3b5df None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:38.207265 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa5ae0f-98d9-4189-a072-b3c396de5811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:38.207920 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5ae0f-98d9-4189-a072-b3c396de5811 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:38.208229 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5ae0f-98d9-4189-a072-b3c396de5811 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:38.208509 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5ae0f-98d9-4189-a072-b3c396de5811 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:38.209295 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa5ae0f-98d9-4189-a072-b3c396de5811 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:38.209596 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa5ae0f-98d9-4189-a072-b3c396de5811 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:38.211000 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa5ae0f-98d9-4189-a072-b3c396de5811 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:38.226208 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 392/784] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:54:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:38.235345 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b2a75-5053-4d6f-a70d-ab7010b06850 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:38.495138 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b2a75-5053-4d6f-a70d-ab7010b06850 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:38.576105 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b004e-9ecb-40cd-8426-e7a04196027a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:38.576803 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b004e-9ecb-40cd-8426-e7a04196027a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:38.577124 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b004e-9ecb-40cd-8426-e7a04196027a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:38.577396 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b004e-9ecb-40cd-8426-e7a04196027a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:38.591511 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3b004e-9ecb-40cd-8426-e7a04196027a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:38.593061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3b004e-9ecb-40cd-8426-e7a04196027a None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:38.624864 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-8f3b004e-9ecb-40cd-8426-e7a04196027a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:38.624864 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:38.624864 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:38.624864 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:38.624864 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:38.624864 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:38.624864 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:38.624864 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:38.624864 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:38.624864 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:38.624864 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:38.624864 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:38.624864 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:38.624864 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:38.624864 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:38.626650 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:38.626650 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:38.626650 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:38.626650 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:38.626650 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:38.626650 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:38.626650 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:38.626650 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:38.626650 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:38.626650 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:38.626650 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:38.626650 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:38.626650 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:38.626650 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:38.626650 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:38.626650 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:38.626650 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:38.628373 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:38.628373 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:38.628373 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:38.628373 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:38.628373 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 393/785] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:54:38.636444 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa5ae0f-98d9-4189-a072-b3c396de5811 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:38.850138 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa5ae0f-98d9-4189-a072-b3c396de5811 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:38.921672 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392285d8-ff4e-4959-9f4f-ba0d47008cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:38.922673 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392285d8-ff4e-4959-9f4f-ba0d47008cfa None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:38.923105 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392285d8-ff4e-4959-9f4f-ba0d47008cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:38.923535 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392285d8-ff4e-4959-9f4f-ba0d47008cfa None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:38.924787 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392285d8-ff4e-4959-9f4f-ba0d47008cfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:38.925267 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392285d8-ff4e-4959-9f4f-ba0d47008cfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:38.927543 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392285d8-ff4e-4959-9f4f-ba0d47008cfa None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:38.931901 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:38.964862 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 393/786] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:54:38.966969 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b004e-9ecb-40cd-8426-e7a04196027a None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:39.000745 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:39.204385 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b004e-9ecb-40cd-8426-e7a04196027a None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:39.278306 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85333c9c-1571-4525-ae24-4c935a3749f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:39.279469 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85333c9c-1571-4525-ae24-4c935a3749f6 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:39.279969 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85333c9c-1571-4525-ae24-4c935a3749f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:39.280500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85333c9c-1571-4525-ae24-4c935a3749f6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:39.297456 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85333c9c-1571-4525-ae24-4c935a3749f6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:39.300196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85333c9c-1571-4525-ae24-4c935a3749f6 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:39.331726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-85333c9c-1571-4525-ae24-4c935a3749f6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 13:54:39.331726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:39.331726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:39.331726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:39.331726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:39.331726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:39.331726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:39.331726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:39.331726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:39.331726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:39.331726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:39.331726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:39.331726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:54:39.331726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:54:39.331726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:54:39.331726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:54:39.333518 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:39.333518 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:39.333518 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:54:39.333518 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:39.333518 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:54:39.333518 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:39.333518 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:39.333518 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:39.333518 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:39.333518 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:39.333518 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:39.333518 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:39.333518 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:39.333518 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:39.333518 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:39.333518 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:39.333518 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:39.333518 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:39.335936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:39.335936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:39.335936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:54:39.335936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:54:39.335936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:54:39.335936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:54:39.335936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:54:39.335936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:54:39.335936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 13:54:39.335936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:39.335936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 394/787] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:54:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:39.346079 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392285d8-ff4e-4959-9f4f-ba0d47008cfa None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:39.530936 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392285d8-ff4e-4959-9f4f-ba0d47008cfa None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:39.588093 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09479b6f-3ce5-4044-b96a-9d6c08a093e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:39.589099 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09479b6f-3ce5-4044-b96a-9d6c08a093e4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:39.589536 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09479b6f-3ce5-4044-b96a-9d6c08a093e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:39.589959 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09479b6f-3ce5-4044-b96a-9d6c08a093e4 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:39.591072 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09479b6f-3ce5-4044-b96a-9d6c08a093e4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:39.591613 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09479b6f-3ce5-4044-b96a-9d6c08a093e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:39.593441 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09479b6f-3ce5-4044-b96a-9d6c08a093e4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:39.622394 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 394/788] 10.222.0.45 () {56 vars in 1135 bytes} [Mon Aug 1 13:54:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:39.634263 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85333c9c-1571-4525-ae24-4c935a3749f6 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:39.670744 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c968d-311b-4bf3-81cf-de4efa91ddf1 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:39.671176 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c968d-311b-4bf3-81cf-de4efa91ddf1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:39.671727 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c968d-311b-4bf3-81cf-de4efa91ddf1 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:39.674355 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 395/789] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:39.698261 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec585bc-83e8-47cd-8425-f42d2b826bf7 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:39.698592 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec585bc-83e8-47cd-8425-f42d2b826bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:39.698832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec585bc-83e8-47cd-8425-f42d2b826bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:39.733368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-aec585bc-83e8-47cd-8425-f42d2b826bf7 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:39.734735 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-aec585bc-83e8-47cd-8425-f42d2b826bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:39.761389 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:39.851177 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:39.877713 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85333c9c-1571-4525-ae24-4c935a3749f6 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:39.907517 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 396/790] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:54:39.932881 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec585bc-83e8-47cd-8425-f42d2b826bf7 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:39.975358 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae1bb28-f4ca-48dc-9c89-34a12c8a2da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:39.976480 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1bb28-f4ca-48dc-9c89-34a12c8a2da9 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:39.976968 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1bb28-f4ca-48dc-9c89-34a12c8a2da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:39.977448 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1bb28-f4ca-48dc-9c89-34a12c8a2da9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:40.014185 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae1bb28-f4ca-48dc-9c89-34a12c8a2da9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:40.015831 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae1bb28-f4ca-48dc-9c89-34a12c8a2da9 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:40.053892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-8ae1bb28-f4ca-48dc-9c89-34a12c8a2da9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:40.053892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:40.053892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:40.053892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:40.053892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:40.053892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:40.053892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:40.053892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:40.053892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:40.053892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:40.053892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:40.053892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:40.053892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:40.053892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:40.053892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:40.055545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:40.055545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:40.055545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:40.055545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:40.055545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:40.055545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:40.055545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:40.055545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:40.055545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:40.055545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:40.055545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:40.055545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:40.055545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:40.055545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:40.055545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:40.055545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:40.055545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:40.057215 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:40.057215 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:40.057215 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:40.057215 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:40.057215 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 395/791] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 423 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:40.065157 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae1bb28-f4ca-48dc-9c89-34a12c8a2da9 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:40.205787 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec585bc-83e8-47cd-8425-f42d2b826bf7 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:40.281186 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb72e30-2db3-4f26-8421-3ab053f14bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:40.281932 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb72e30-2db3-4f26-8421-3ab053f14bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:40.282219 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb72e30-2db3-4f26-8421-3ab053f14bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:40.282510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb72e30-2db3-4f26-8421-3ab053f14bf2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:40.283389 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb72e30-2db3-4f26-8421-3ab053f14bf2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:40.284902 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb72e30-2db3-4f26-8421-3ab053f14bf2 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:40.314075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-eeb72e30-2db3-4f26-8421-3ab053f14bf2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 13:54:40.314075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:40.314075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:54:40.314075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:54:40.314075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:54:40.314075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:54:40.314075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:54:40.314075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:54:40.314075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 13:54:40.314075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:40.314075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:54:40.314075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:40.314075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:40.314075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:40.314075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:40.314075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:40.315810 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:40.315810 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:40.315810 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:40.315810 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:40.315810 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:40.315810 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:40.315810 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:40.315810 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:54:40.315810 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:54:40.315810 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:54:40.315810 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:54:40.315810 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:40.315810 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:40.315810 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:40.315810 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:40.315810 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:40.317351 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:40.317351 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:40.317351 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:40.317351 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:40.317351 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:40.317351 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:40.317351 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:40.317351 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:40.317351 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:40.317351 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:40.317351 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:40.317351 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:54:40.317351 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:54:40.317351 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 13:54:40.317351 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:40.326682 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 397/792] 10.222.0.45 () {56 vars in 1129 bytes} [Mon Aug 1 13:54:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:54:40.331662 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae1bb28-f4ca-48dc-9c89-34a12c8a2da9 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:40.333508 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb72e30-2db3-4f26-8421-3ab053f14bf2 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:40.372766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:40.421177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d1a51-73d6-4d31-b7ad-5fa4704d55f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:40.421915 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d1a51-73d6-4d31-b7ad-5fa4704d55f0 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:40.422204 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d1a51-73d6-4d31-b7ad-5fa4704d55f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:40.422469 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d1a51-73d6-4d31-b7ad-5fa4704d55f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:40.423193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d1a51-73d6-4d31-b7ad-5fa4704d55f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:40.423503 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d1a51-73d6-4d31-b7ad-5fa4704d55f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:40.425222 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d1a51-73d6-4d31-b7ad-5fa4704d55f0 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:40.464354 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 396/793] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:40.478013 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d1a51-73d6-4d31-b7ad-5fa4704d55f0 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:40.485644 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:40.583905 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb72e30-2db3-4f26-8421-3ab053f14bf2 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:40.633347 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6fdf29-3e32-4bff-9126-c68fd00ce1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:40.635043 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6fdf29-3e32-4bff-9126-c68fd00ce1ac None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:40.635673 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6fdf29-3e32-4bff-9126-c68fd00ce1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:40.635958 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6fdf29-3e32-4bff-9126-c68fd00ce1ac None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:40.637168 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6fdf29-3e32-4bff-9126-c68fd00ce1ac None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:40.637586 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6fdf29-3e32-4bff-9126-c68fd00ce1ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:40.640740 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6fdf29-3e32-4bff-9126-c68fd00ce1ac None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:40.648781 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 398/794] 10.222.0.45 () {56 vars in 1124 bytes} [Mon Aug 1 13:54:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:40.650577 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b7e431-d033-4394-a187-e9bdc2ecb6ab None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:40.650877 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b7e431-d033-4394-a187-e9bdc2ecb6ab None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:40.651178 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b7e431-d033-4394-a187-e9bdc2ecb6ab None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:40.652931 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 399/795] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:40.655671 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a9690-585a-4ecd-a485-54f822816034 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:40.655671 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a9690-585a-4ecd-a485-54f822816034 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:40.655671 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a9690-585a-4ecd-a485-54f822816034 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:40.663420 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 400/796] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:40.663420 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a9690-585a-4ecd-a485-54f822816034 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:40.721873 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d1a51-73d6-4d31-b7ad-5fa4704d55f0 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:40.769149 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66c70a-7e32-42fc-9c2a-f4b1d4fe5a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:40.770527 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66c70a-7e32-42fc-9c2a-f4b1d4fe5a51 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:40.770881 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66c70a-7e32-42fc-9c2a-f4b1d4fe5a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:40.771282 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66c70a-7e32-42fc-9c2a-f4b1d4fe5a51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:40.782131 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc66c70a-7e32-42fc-9c2a-f4b1d4fe5a51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:40.784697 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc66c70a-7e32-42fc-9c2a-f4b1d4fe5a51 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:40.802526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-dc66c70a-7e32-42fc-9c2a-f4b1d4fe5a51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:40.802526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:40.802526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:40.802526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:40.802526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:40.802526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:40.802526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:40.802526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:40.802526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:40.802526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:40.802526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:40.802526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:40.802526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:54:40.802526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:54:40.802526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:54:40.804722 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:54:40.804722 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:40.804722 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:40.804722 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:40.804722 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:40.804722 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:40.804722 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:40.804722 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:40.804722 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:40.804722 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:40.804722 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:40.804722 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:40.804722 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:40.804722 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:40.804722 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:40.804722 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:40.804722 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:40.806585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:54:40.806585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:54:40.806585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:54:40.806585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:54:40.806585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:54:40.806585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:54:40.806585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:40.806585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:40.811504 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 397/797] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:54:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:40.813064 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e43a4-fbb6-46b3-9ea4-195ebd34d6d3 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:40.813369 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e43a4-fbb6-46b3-9ea4-195ebd34d6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:40.813609 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e43a4-fbb6-46b3-9ea4-195ebd34d6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:40.827371 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a9690-585a-4ecd-a485-54f822816034 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:40.834521 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-ec3e43a4-fbb6-46b3-9ea4-195ebd34d6d3 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:40.835829 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ec3e43a4-fbb6-46b3-9ea4-195ebd34d6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:40.845753 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:40.873130 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:40.877644 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14858abd-3209-4309-a25b-5c8a46cefd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:40.878590 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14858abd-3209-4309-a25b-5c8a46cefd3b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:40.878875 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14858abd-3209-4309-a25b-5c8a46cefd3b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:40.879163 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14858abd-3209-4309-a25b-5c8a46cefd3b None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:40.879875 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14858abd-3209-4309-a25b-5c8a46cefd3b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:40.880162 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14858abd-3209-4309-a25b-5c8a46cefd3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:40.881587 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14858abd-3209-4309-a25b-5c8a46cefd3b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:40.887317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 401/798] 10.222.0.45 () {56 vars in 1124 bytes} [Mon Aug 1 13:54:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:40.888624 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb251771-1f14-4af9-954b-75b9c33c0929 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:40.888906 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb251771-1f14-4af9-954b-75b9c33c0929 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:40.889183 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb251771-1f14-4af9-954b-75b9c33c0929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:40.912575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-eb251771-1f14-4af9-954b-75b9c33c0929 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:40.913926 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-eb251771-1f14-4af9-954b-75b9c33c0929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:40.942267 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 398/799] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:54:40.949328 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e43a4-fbb6-46b3-9ea4-195ebd34d6d3 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:41.016041 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 402/800] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:54:41.017860 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb251771-1f14-4af9-954b-75b9c33c0929 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:41.126746 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e43a4-fbb6-46b3-9ea4-195ebd34d6d3 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:41.170954 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb251771-1f14-4af9-954b-75b9c33c0929 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:41.182602 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:41.185030 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ef076-113b-4d86-a13c-b46c171d20be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:41.185931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ef076-113b-4d86-a13c-b46c171d20be None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:41.186407 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ef076-113b-4d86-a13c-b46c171d20be None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:41.186875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ef076-113b-4d86-a13c-b46c171d20be None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:41.187876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ef076-113b-4d86-a13c-b46c171d20be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:41.188225 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ef076-113b-4d86-a13c-b46c171d20be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:41.189904 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ef076-113b-4d86-a13c-b46c171d20be None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:41.206659 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 399/801] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:54:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:41.206659 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ef076-113b-4d86-a13c-b46c171d20be None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:41.233513 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4511eb-c325-4bd5-8c6c-d80364977a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:41.234316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4511eb-c325-4bd5-8c6c-d80364977a94 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:41.234772 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4511eb-c325-4bd5-8c6c-d80364977a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:41.235207 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4511eb-c325-4bd5-8c6c-d80364977a94 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:41.236463 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4511eb-c325-4bd5-8c6c-d80364977a94 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:41.241240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4511eb-c325-4bd5-8c6c-d80364977a94 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:41.247827 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:41.279430 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 403/802] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 13:54:41 2022] POST /identity/v3/endpoints => generated 346 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:54:41.279430 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4511eb-c325-4bd5-8c6c-d80364977a94 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:41.372514 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ef076-113b-4d86-a13c-b46c171d20be None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:41.430638 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa7f19-c266-4da5-bf9a-1050cfe0899e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:41.431300 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa7f19-c266-4da5-bf9a-1050cfe0899e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:41.431590 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa7f19-c266-4da5-bf9a-1050cfe0899e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:41.431849 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa7f19-c266-4da5-bf9a-1050cfe0899e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:41.447183 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa7f19-c266-4da5-bf9a-1050cfe0899e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:54:41.453907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-bafa7f19-c266-4da5-bf9a-1050cfe0899e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:54:41.453907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:41.453907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:41.453907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:41.453907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:41.453907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:41.453907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:41.453907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:41.453907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:41.453907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:41.453907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:41.453907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:41.453907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:54:41.453907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:54:41.453907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:54:41.453907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:54:41.455815 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:54:41.455815 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:54:41.455815 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:54:41.455815 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:54:41.455815 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:41.455815 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:41.455815 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:41.455815 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:41.455815 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:41.455815 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:41.455815 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:41.455815 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:41.455815 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:41.455815 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:41.455815 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:41.455815 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:41.457574 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:41.457574 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:41.457574 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:41.457574 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:41.457574 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:54:41.457574 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:54:41.457574 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:54:41.457574 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:54:41.457574 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:54:41.457574 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:54:41.457574 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:54:41.457574 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:41.457574 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 400/803] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 13:54:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:41.457574 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db15b4-7295-4ab2-bede-ff6a288f12b9 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:41.459137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db15b4-7295-4ab2-bede-ff6a288f12b9 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:41.459137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db15b4-7295-4ab2-bede-ff6a288f12b9 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:41.461286 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 401/804] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:41.462647 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db15b4-7295-4ab2-bede-ff6a288f12b9 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:41.491200 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4511eb-c325-4bd5-8c6c-d80364977a94 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:41.570462 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463a2a56-8544-40bd-ad69-88a49252d19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:41.571953 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a2a56-8544-40bd-ad69-88a49252d19f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:41.572641 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a2a56-8544-40bd-ad69-88a49252d19f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:41.573161 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a2a56-8544-40bd-ad69-88a49252d19f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:41.599060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463a2a56-8544-40bd-ad69-88a49252d19f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:54:41.611934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-463a2a56-8544-40bd-ad69-88a49252d19f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:54:41.611934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:41.611934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:41.611934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:41.611934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:41.611934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:41.611934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:41.611934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:41.611934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:41.611934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:41.611934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:41.611934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:41.611934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:54:41.611934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:54:41.611934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:54:41.611934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:54:41.613342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:54:41.613342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:54:41.613342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:54:41.613342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:54:41.613342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:41.613342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:41.613342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:41.613342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:41.613342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:41.613342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:41.613342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:41.613342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:41.613342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:41.613342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:41.613342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:41.613342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:41.614256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:41.614256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:41.614256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:41.614256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:41.614256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:54:41.614256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:54:41.614256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:54:41.614256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:54:41.614256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:54:41.614256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:54:41.614256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:54:41.614256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:41.615867 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 404/805] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:54:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:41.620452 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463a2a56-8544-40bd-ad69-88a49252d19f None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:41.650791 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:41.673300 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:41.690167 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db15b4-7295-4ab2-bede-ff6a288f12b9 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:41.756289 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a999b-dcfe-4dcb-aa1c-e47f6be3a2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:41.757564 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a999b-dcfe-4dcb-aa1c-e47f6be3a2a7 None admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:41.758044 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a999b-dcfe-4dcb-aa1c-e47f6be3a2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:41.758535 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a999b-dcfe-4dcb-aa1c-e47f6be3a2a7 None admin] PATH_INFO: `/v3/projects/36f063d8f65d494a89dbbd72c53417c8/users/0cda7f365ba0405daaf86870960f12d2/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:41.762478 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92a999b-dcfe-4dcb-aa1c-e47f6be3a2a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=36f063d8f65d494a89dbbd72c53417c8, user_id=0cda7f365ba0405daaf86870960f12d2, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:41.765575 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92a999b-dcfe-4dcb-aa1c-e47f6be3a2a7 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:41.800401 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 402/806] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 13:54:41 2022] PUT /identity/v3/projects/36f063d8f65d494a89dbbd72c53417c8/users/0cda7f365ba0405daaf86870960f12d2/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:54:41.800401 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3fef59-1a51-482d-a4ec-8ac0d6e2e669 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:41.800401 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3fef59-1a51-482d-a4ec-8ac0d6e2e669 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:41.800401 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3fef59-1a51-482d-a4ec-8ac0d6e2e669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:41.837718 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-2c3fef59-1a51-482d-a4ec-8ac0d6e2e669 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:41.840052 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-2c3fef59-1a51-482d-a4ec-8ac0d6e2e669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:41.859542 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463a2a56-8544-40bd-ad69-88a49252d19f None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:41.944182 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1356fcc8-d30d-4eba-83ed-6858ec1ff64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:41.945622 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356fcc8-d30d-4eba-83ed-6858ec1ff64f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:41.946075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356fcc8-d30d-4eba-83ed-6858ec1ff64f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:41.946528 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356fcc8-d30d-4eba-83ed-6858ec1ff64f None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:41.947807 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1356fcc8-d30d-4eba-83ed-6858ec1ff64f None admin] RBAC: Adding query filter params (name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:41.948736 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1356fcc8-d30d-4eba-83ed-6858ec1ff64f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:41.952178 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1356fcc8-d30d-4eba-83ed-6858ec1ff64f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:41.970953 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 405/807] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 13:54:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:41.973176 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1356fcc8-d30d-4eba-83ed-6858ec1ff64f None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:41.998795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 403/808] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:54:42.000741 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3fef59-1a51-482d-a4ec-8ac0d6e2e669 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:42.181425 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:42.196916 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:42.219452 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1356fcc8-d30d-4eba-83ed-6858ec1ff64f None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:42.227725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3fef59-1a51-482d-a4ec-8ac0d6e2e669 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:42.293459 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ab3a8-ddd7-4e65-bbdd-af2314274b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:42.294799 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ab3a8-ddd7-4e65-bbdd-af2314274b3d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:42.295286 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ab3a8-ddd7-4e65-bbdd-af2314274b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:42.295746 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ab3a8-ddd7-4e65-bbdd-af2314274b3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:42.297662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b58df-1956-498c-8151-93216cc3d9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:42.299006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b58df-1956-498c-8151-93216cc3d9dd None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:42.300060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b58df-1956-498c-8151-93216cc3d9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:42.300744 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b58df-1956-498c-8151-93216cc3d9dd None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:42.302844 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b58df-1956-498c-8151-93216cc3d9dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:42.303627 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b58df-1956-498c-8151-93216cc3d9dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:42.306397 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b58df-1956-498c-8151-93216cc3d9dd None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:42.316918 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162ab3a8-ddd7-4e65-bbdd-af2314274b3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:42.319578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162ab3a8-ddd7-4e65-bbdd-af2314274b3d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:42.333050 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 406/809] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:54:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:42.336219 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b58df-1956-498c-8151-93216cc3d9dd None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:42.352995 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-162ab3a8-ddd7-4e65-bbdd-af2314274b3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:42.352995 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:42.352995 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:42.352995 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:42.352995 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:42.352995 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:42.352995 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:42.352995 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:42.352995 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:42.352995 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:42.352995 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:42.352995 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:42.352995 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:42.352995 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:42.352995 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:42.354253 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:42.354253 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:42.354253 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:42.354253 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:42.354253 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:42.354253 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:42.354253 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:42.354253 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:42.354253 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:42.354253 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:42.354253 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:42.354253 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:42.354253 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:42.354253 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:42.354253 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:42.354253 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:42.354253 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:42.355931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:42.355931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:42.355931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:42.355931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:42.356383 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 404/810] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:42.357484 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ab3a8-ddd7-4e65-bbdd-af2314274b3d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:42.485472 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b58df-1956-498c-8151-93216cc3d9dd None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:42.534278 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af98a55f-7e53-4ddb-8fd6-27a79298a7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:42.535515 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af98a55f-7e53-4ddb-8fd6-27a79298a7a0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:42.536042 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af98a55f-7e53-4ddb-8fd6-27a79298a7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:42.536480 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af98a55f-7e53-4ddb-8fd6-27a79298a7a0 None admin] PATH_INFO: `/v3/services/network` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:42.537624 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af98a55f-7e53-4ddb-8fd6-27a79298a7a0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:42.540202 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af98a55f-7e53-4ddb-8fd6-27a79298a7a0 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:42.550698 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ab3a8-ddd7-4e65-bbdd-af2314274b3d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:42.558238 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-af98a55f-7e53-4ddb-8fd6-27a79298a7a0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 13:54:42.558238 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:42.558238 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:54:42.558238 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:54:42.558238 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:54:42.558238 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:54:42.558238 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:54:42.558238 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:54:42.558238 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 13:54:42.558238 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:42.558238 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:54:42.558238 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:42.558238 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:42.558238 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:42.558238 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:42.558238 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:42.558238 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:42.559424 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:42.559424 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:42.559424 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:42.559424 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:42.559424 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:42.559424 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:42.559424 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:54:42.559424 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:54:42.559424 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:54:42.559424 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:54:42.559424 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:42.559424 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:42.559424 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:42.559424 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:42.559424 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:42.559424 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:42.561161 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:42.561161 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:42.561161 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:42.561161 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:42.561161 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:42.561161 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:42.561161 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:42.561161 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:42.561161 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:42.561161 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:42.561161 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:54:42.561161 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:54:42.561161 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 13:54:42.561161 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:42.561161 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 407/811] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:54:42 2022] GET /identity/v3/services/network => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:42.562222 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af98a55f-7e53-4ddb-8fd6-27a79298a7a0 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:42.582627 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:42.590297 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:42.605108 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53dbe6-93f9-4104-b7a2-1ea1c6e576f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:42.606592 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53dbe6-93f9-4104-b7a2-1ea1c6e576f0 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:42.607284 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53dbe6-93f9-4104-b7a2-1ea1c6e576f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:42.607917 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53dbe6-93f9-4104-b7a2-1ea1c6e576f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:42.627769 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc53dbe6-93f9-4104-b7a2-1ea1c6e576f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:42.631077 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc53dbe6-93f9-4104-b7a2-1ea1c6e576f0 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:42.658068 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-cc53dbe6-93f9-4104-b7a2-1ea1c6e576f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:42.658068 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:42.658068 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:42.658068 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:42.658068 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:42.658068 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:42.658068 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:42.658068 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:42.658068 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:42.658068 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:42.658068 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:42.658068 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:42.658068 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:42.658068 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:42.658068 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:42.659809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:42.659809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:42.659809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:42.659809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:42.659809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:42.659809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:42.659809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:42.659809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:42.659809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:42.659809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:42.659809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:42.659809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:42.659809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:42.659809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:42.659809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:42.659809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:42.659809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:42.661164 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:42.661164 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:42.661164 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:42.661164 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:42.669019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 405/812] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:42.670370 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53dbe6-93f9-4104-b7a2-1ea1c6e576f0 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:42.754551 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af98a55f-7e53-4ddb-8fd6-27a79298a7a0 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:42.821502 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7bd85-cada-4e76-87a9-111aec345e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:42.822469 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7bd85-cada-4e76-87a9-111aec345e6c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:42.822898 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7bd85-cada-4e76-87a9-111aec345e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:42.823316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7bd85-cada-4e76-87a9-111aec345e6c None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:42.824530 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e7bd85-cada-4e76-87a9-111aec345e6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:42.825007 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e7bd85-cada-4e76-87a9-111aec345e6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:42.825744 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53dbe6-93f9-4104-b7a2-1ea1c6e576f0 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:42.827364 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e7bd85-cada-4e76-87a9-111aec345e6c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:42.849651 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 408/813] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:42.851143 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7bd85-cada-4e76-87a9-111aec345e6c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:42.883309 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad4b0b-cc5f-4724-940e-1239452af2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:42.884802 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad4b0b-cc5f-4724-940e-1239452af2c6 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:42.885488 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad4b0b-cc5f-4724-940e-1239452af2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:42.886181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad4b0b-cc5f-4724-940e-1239452af2c6 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:42.887526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad4b0b-cc5f-4724-940e-1239452af2c6 None admin] RBAC: Adding query filter params (name=network) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:42.888264 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad4b0b-cc5f-4724-940e-1239452af2c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:42.890880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad4b0b-cc5f-4724-940e-1239452af2c6 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:42.899692 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 406/814] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Aug 1 13:54:42 2022] GET /identity/v3/services?name=network => generated 124 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:42.901525 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad4b0b-cc5f-4724-940e-1239452af2c6 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:42.955467 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:42.978128 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:43.000684 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7bd85-cada-4e76-87a9-111aec345e6c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:43.067089 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84767e-7e9d-4b9a-972f-48a0364254ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:43.067356 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84767e-7e9d-4b9a-972f-48a0364254ef None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:43.067686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84767e-7e9d-4b9a-972f-48a0364254ef None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:43.067833 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84767e-7e9d-4b9a-972f-48a0364254ef None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:43.068580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d84767e-7e9d-4b9a-972f-48a0364254ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:43.068903 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d84767e-7e9d-4b9a-972f-48a0364254ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:43.071356 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d84767e-7e9d-4b9a-972f-48a0364254ef None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:43.092340 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 409/815] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:43.092340 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84767e-7e9d-4b9a-972f-48a0364254ef None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:43.099755 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad4b0b-cc5f-4724-940e-1239452af2c6 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:43.182040 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3874fac-c173-4a2b-81e0-fac383c769b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:43.183192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3874fac-c173-4a2b-81e0-fac383c769b2 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:43.183699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3874fac-c173-4a2b-81e0-fac383c769b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:43.184232 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3874fac-c173-4a2b-81e0-fac383c769b2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:43.197754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3874fac-c173-4a2b-81e0-fac383c769b2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:43.200080 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3874fac-c173-4a2b-81e0-fac383c769b2 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:43.221223 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-e3874fac-c173-4a2b-81e0-fac383c769b2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 13:54:43.221223 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:43.221223 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:43.221223 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:43.221223 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:43.221223 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:43.221223 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:43.221223 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:43.221223 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:43.221223 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:43.221223 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:43.221223 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:43.221223 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:54:43.221223 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:54:43.221223 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:54:43.221223 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:54:43.223302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:43.223302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:43.223302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:54:43.223302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:43.223302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:54:43.223302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:43.223302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:43.223302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:43.223302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:43.223302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:43.223302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:43.223302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:43.223302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:43.223302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:43.223302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:43.223302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:43.223302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:43.223302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:43.225389 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:43.225389 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:43.225389 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:54:43.225389 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:54:43.225389 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:54:43.225389 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:54:43.225389 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:54:43.225389 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:54:43.225389 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 13:54:43.225389 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:43.228641 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 407/816] 10.222.0.45 () {56 vars in 1117 bytes} [Mon Aug 1 13:54:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:43.232416 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3874fac-c173-4a2b-81e0-fac383c769b2 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:43.249172 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84767e-7e9d-4b9a-972f-48a0364254ef None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:43.290695 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:43.299738 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e537ca1-128f-4d8c-a7cf-66211223f141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:43.301182 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e537ca1-128f-4d8c-a7cf-66211223f141 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:43.301908 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e537ca1-128f-4d8c-a7cf-66211223f141 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:43.302752 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e537ca1-128f-4d8c-a7cf-66211223f141 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:43.304361 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e537ca1-128f-4d8c-a7cf-66211223f141 None admin] RBAC: Adding query filter params (type=network) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:43.305229 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e537ca1-128f-4d8c-a7cf-66211223f141 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:43.307029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e537ca1-128f-4d8c-a7cf-66211223f141 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:43.316075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 410/817] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Aug 1 13:54:43 2022] GET /identity/v3/services?type=network => generated 350 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:43.318240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e537ca1-128f-4d8c-a7cf-66211223f141 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:43.417584 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3874fac-c173-4a2b-81e0-fac383c769b2 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:43.420841 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:43.484896 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77330086-c353-4ef7-a4b4-28ef8d52458d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:43.486052 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77330086-c353-4ef7-a4b4-28ef8d52458d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:43.486550 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77330086-c353-4ef7-a4b4-28ef8d52458d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:43.487032 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77330086-c353-4ef7-a4b4-28ef8d52458d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:43.503295 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77330086-c353-4ef7-a4b4-28ef8d52458d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:43.505967 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77330086-c353-4ef7-a4b4-28ef8d52458d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:43.516815 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e537ca1-128f-4d8c-a7cf-66211223f141 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:43.528196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-77330086-c353-4ef7-a4b4-28ef8d52458d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:54:43.528196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:43.528196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:43.528196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:43.528196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:43.528196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:43.528196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:43.528196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:43.528196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:43.528196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:43.528196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:43.528196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:43.528196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:54:43.528196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:54:43.528196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:54:43.528196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:54:43.530017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:43.530017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:43.530017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:54:43.530017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:43.530017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:54:43.530017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:43.530017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:43.530017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:43.530017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:43.530017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:43.530017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:43.530017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:43.530017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:43.530017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:43.530017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:43.530017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:43.530017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:43.530017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:43.531903 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:43.531903 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:43.531903 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:54:43.531903 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:54:43.531903 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:54:43.531903 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:54:43.531903 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:54:43.531903 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:54:43.531903 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:54:43.531903 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:43.536989 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 408/818] 10.222.0.45 () {56 vars in 1093 bytes} [Mon Aug 1 13:54:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:43.538655 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77330086-c353-4ef7-a4b4-28ef8d52458d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:43.601363 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7737e-5536-4d8f-912b-663e636eff79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:43.602911 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7737e-5536-4d8f-912b-663e636eff79 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:43.603681 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7737e-5536-4d8f-912b-663e636eff79 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:43.604461 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7737e-5536-4d8f-912b-663e636eff79 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:43.606524 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7737e-5536-4d8f-912b-663e636eff79 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:43.607379 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7737e-5536-4d8f-912b-663e636eff79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:43.610690 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7737e-5536-4d8f-912b-663e636eff79 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:43.650184 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 411/819] 10.222.0.45 () {56 vars in 1149 bytes} [Mon Aug 1 13:54:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:43.652153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7737e-5536-4d8f-912b-663e636eff79 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:43.760924 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77330086-c353-4ef7-a4b4-28ef8d52458d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:43.763913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:43.829350 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0097bfde-3ef4-406e-b0d0-005627e11cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:43.830569 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097bfde-3ef4-406e-b0d0-005627e11cd3 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:43.831113 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097bfde-3ef4-406e-b0d0-005627e11cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:43.831707 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097bfde-3ef4-406e-b0d0-005627e11cd3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:43.833149 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0097bfde-3ef4-406e-b0d0-005627e11cd3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:43.835784 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0097bfde-3ef4-406e-b0d0-005627e11cd3 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:43.866877 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7737e-5536-4d8f-912b-663e636eff79 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:43.882361 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 409/820] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 13:54:43 2022] POST /identity/v3/endpoints => generated 327 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:54:43.884089 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0097bfde-3ef4-406e-b0d0-005627e11cd3 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:43.925621 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:43.955136 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb565b4-44e1-4a67-a213-2e291c18552a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:43.956703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb565b4-44e1-4a67-a213-2e291c18552a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:43.957467 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb565b4-44e1-4a67-a213-2e291c18552a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:43.958172 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb565b4-44e1-4a67-a213-2e291c18552a None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:43.959992 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb565b4-44e1-4a67-a213-2e291c18552a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:43.960941 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb565b4-44e1-4a67-a213-2e291c18552a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:43.964388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb565b4-44e1-4a67-a213-2e291c18552a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:44.002330 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 412/821] 10.222.0.45 () {56 vars in 1133 bytes} [Mon Aug 1 13:54:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:44.012256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb565b4-44e1-4a67-a213-2e291c18552a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:44.103085 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0097bfde-3ef4-406e-b0d0-005627e11cd3 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:44.183526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b9dc0-7436-4485-aa26-b42595c6ff12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:44.184668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b9dc0-7436-4485-aa26-b42595c6ff12 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:44.185182 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b9dc0-7436-4485-aa26-b42595c6ff12 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:44.185674 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b9dc0-7436-4485-aa26-b42595c6ff12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:44.230325 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0b9dc0-7436-4485-aa26-b42595c6ff12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:44.233328 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0b9dc0-7436-4485-aa26-b42595c6ff12 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:44.243043 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb565b4-44e1-4a67-a213-2e291c18552a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:44.259383 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:44.271143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-da0b9dc0-7436-4485-aa26-b42595c6ff12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:44.271143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:44.271143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:44.271143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:44.271143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:44.271143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:44.271143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:44.271143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:44.271143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:44.271143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:44.271143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:44.271143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:44.271143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:44.271143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:44.271143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:44.272871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:44.272871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:44.272871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:44.272871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:44.272871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:44.272871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:44.272871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:44.272871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:44.272871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:44.272871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:44.272871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:44.272871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:44.272871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:44.272871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:44.272871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:44.272871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:44.272871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:44.274666 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:44.274666 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:44.274666 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:44.274666 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:44.278982 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 410/822] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:44.287611 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b9dc0-7436-4485-aa26-b42595c6ff12 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:44.359017 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367aa51d-2dcb-4e67-9847-4dcc97df0fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:44.362040 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367aa51d-2dcb-4e67-9847-4dcc97df0fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:44.362961 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367aa51d-2dcb-4e67-9847-4dcc97df0fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:44.363824 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367aa51d-2dcb-4e67-9847-4dcc97df0fa3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:44.471209 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:44.553230 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367aa51d-2dcb-4e67-9847-4dcc97df0fa3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:44.556053 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367aa51d-2dcb-4e67-9847-4dcc97df0fa3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:44.585392 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b9dc0-7436-4485-aa26-b42595c6ff12 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:44.665063 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8998196e-e7ac-45df-b568-62a494638db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:44.667146 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8998196e-e7ac-45df-b568-62a494638db6 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:44.667991 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8998196e-e7ac-45df-b568-62a494638db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:44.668851 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8998196e-e7ac-45df-b568-62a494638db6 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:44.671131 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8998196e-e7ac-45df-b568-62a494638db6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:44.672746 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8998196e-e7ac-45df-b568-62a494638db6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:44.675439 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8998196e-e7ac-45df-b568-62a494638db6 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:44.697626 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 411/823] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:44.699985 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64fd58-ca8e-4484-bc52-e4ce8f3c6d32 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:44.700523 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64fd58-ca8e-4484-bc52-e4ce8f3c6d32 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:44.701007 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64fd58-ca8e-4484-bc52-e4ce8f3c6d32 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:44.703849 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 412/824] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:44.710411 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64fd58-ca8e-4484-bc52-e4ce8f3c6d32 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:44.789917 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 413/825] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:54:44 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 779 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 13:54:44.792402 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2324a67-62ed-483f-904d-4ef96ee78d36 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:44.793256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2324a67-62ed-483f-904d-4ef96ee78d36 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:44.793735 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2324a67-62ed-483f-904d-4ef96ee78d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:44.818712 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:44.829763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-e2324a67-62ed-483f-904d-4ef96ee78d36 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:44.831417 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e2324a67-62ed-483f-904d-4ef96ee78d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:44.917944 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64fd58-ca8e-4484-bc52-e4ce8f3c6d32 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:44.954714 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 414/826] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:54:44.957181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2324a67-62ed-483f-904d-4ef96ee78d36 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:44.981720 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:44.990332 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6e499-182a-4b69-bd7a-23fb6132adfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:44.991532 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6e499-182a-4b69-bd7a-23fb6132adfb None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:44.992031 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6e499-182a-4b69-bd7a-23fb6132adfb None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:44.992569 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6e499-182a-4b69-bd7a-23fb6132adfb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:45.006892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a6e499-182a-4b69-bd7a-23fb6132adfb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:45.012932 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a6e499-182a-4b69-bd7a-23fb6132adfb None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:45.043945 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-61a6e499-182a-4b69-bd7a-23fb6132adfb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:45.043945 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:45.043945 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:45.043945 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:45.043945 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:45.043945 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:45.043945 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:45.043945 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:45.043945 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:45.043945 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:45.043945 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:45.043945 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:45.043945 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:54:45.043945 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:54:45.043945 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:54:45.045000 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:54:45.045000 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:45.045000 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:45.045000 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:45.045000 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:45.045000 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:45.045000 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:45.045000 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:45.045000 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:45.045000 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:45.045000 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:45.045000 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:45.045000 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:45.045000 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:45.045000 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:45.045000 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:45.045000 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:45.045915 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:54:45.045915 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:54:45.045915 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:54:45.045915 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:54:45.045915 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:54:45.045915 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:54:45.045915 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:45.045915 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:45.046536 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 413/827] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:54:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:45.047940 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6e499-182a-4b69-bd7a-23fb6132adfb None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:45.136044 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2324a67-62ed-483f-904d-4ef96ee78d36 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:45.202666 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281588d0-4242-4ff3-9ff8-46f7c20bd624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:45.203745 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281588d0-4242-4ff3-9ff8-46f7c20bd624 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:45.204503 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281588d0-4242-4ff3-9ff8-46f7c20bd624 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:45.205264 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281588d0-4242-4ff3-9ff8-46f7c20bd624 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:45.211450 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:45.221617 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6e499-182a-4b69-bd7a-23fb6132adfb None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:45.229849 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281588d0-4242-4ff3-9ff8-46f7c20bd624 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:45.231786 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281588d0-4242-4ff3-9ff8-46f7c20bd624 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:45.258272 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-281588d0-4242-4ff3-9ff8-46f7c20bd624 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:45.258272 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:45.258272 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:45.258272 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:45.258272 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:45.258272 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:45.258272 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:45.258272 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:45.258272 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:45.258272 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:45.258272 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:45.258272 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:45.258272 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:45.258272 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:45.258272 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:45.269297 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:45.269297 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:45.269297 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:45.269297 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:45.269297 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:45.269297 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:45.269297 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:45.269297 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:45.269297 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:45.269297 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:45.269297 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:45.269297 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:45.269297 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:45.269297 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:45.269297 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:45.269297 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:45.269297 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:45.271766 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:45.271766 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:45.271766 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:45.271766 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:45.271766 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 415/828] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:45.271766 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281588d0-4242-4ff3-9ff8-46f7c20bd624 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:45.302803 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad16ec3-6481-48dd-8c8d-eda013c1027b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:45.303551 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad16ec3-6481-48dd-8c8d-eda013c1027b None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:45.303897 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad16ec3-6481-48dd-8c8d-eda013c1027b None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:45.304115 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad16ec3-6481-48dd-8c8d-eda013c1027b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:45.305032 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad16ec3-6481-48dd-8c8d-eda013c1027b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:45.306824 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad16ec3-6481-48dd-8c8d-eda013c1027b None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:45.326716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-cad16ec3-6481-48dd-8c8d-eda013c1027b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 13:54:45.326716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:45.326716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:54:45.326716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:54:45.326716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:54:45.326716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:54:45.326716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:54:45.326716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:54:45.326716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 13:54:45.326716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:45.326716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:54:45.326716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:45.326716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:45.326716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:45.326716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:45.326716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:45.326716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:45.328395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:45.328395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:45.328395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:45.328395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:45.328395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:45.328395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:45.328395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:54:45.328395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:54:45.328395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:54:45.328395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:54:45.328395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:45.328395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:45.328395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:45.328395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:45.328395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:45.328395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:45.329393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:45.329393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:45.329393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:45.329393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:45.329393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:45.329393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:45.329393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:45.329393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:45.329393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:45.329393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:45.329393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:54:45.329393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:54:45.329393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 13:54:45.329393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:45.331323 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 414/829] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 13:54:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:45.332083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad16ec3-6481-48dd-8c8d-eda013c1027b None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:45.437711 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:45.476524 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281588d0-4242-4ff3-9ff8-46f7c20bd624 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:45.522287 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad16ec3-6481-48dd-8c8d-eda013c1027b None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:45.537070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b888ecc0-b884-416a-aa73-55bca7bf251c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:45.538079 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888ecc0-b884-416a-aa73-55bca7bf251c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:45.538548 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888ecc0-b884-416a-aa73-55bca7bf251c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:45.539571 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888ecc0-b884-416a-aa73-55bca7bf251c None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:45.540836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b888ecc0-b884-416a-aa73-55bca7bf251c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:45.541336 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b888ecc0-b884-416a-aa73-55bca7bf251c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:45.543797 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b888ecc0-b884-416a-aa73-55bca7bf251c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:45.567317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 416/830] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:54:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:45.568577 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b888ecc0-b884-416a-aa73-55bca7bf251c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:45.595075 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d04c7-edf7-4c1b-a495-c8fdc6c936d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:45.596887 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d04c7-edf7-4c1b-a495-c8fdc6c936d3 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:45.596887 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d04c7-edf7-4c1b-a495-c8fdc6c936d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:45.597296 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d04c7-edf7-4c1b-a495-c8fdc6c936d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:45.597923 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d04c7-edf7-4c1b-a495-c8fdc6c936d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:45.598274 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d04c7-edf7-4c1b-a495-c8fdc6c936d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:45.600766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d04c7-edf7-4c1b-a495-c8fdc6c936d3 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:45.624578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 415/831] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:45.624578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d04c7-edf7-4c1b-a495-c8fdc6c936d3 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:45.679176 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:45.763763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b888ecc0-b884-416a-aa73-55bca7bf251c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:45.822286 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec65361d-c2a2-48a8-a66e-df5a86dc0deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:45.822992 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65361d-c2a2-48a8-a66e-df5a86dc0deb None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:45.823297 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65361d-c2a2-48a8-a66e-df5a86dc0deb None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:45.823577 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65361d-c2a2-48a8-a66e-df5a86dc0deb None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:45.824342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec65361d-c2a2-48a8-a66e-df5a86dc0deb None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:45.824695 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec65361d-c2a2-48a8-a66e-df5a86dc0deb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:45.826495 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec65361d-c2a2-48a8-a66e-df5a86dc0deb None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:45.830926 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d04c7-edf7-4c1b-a495-c8fdc6c936d3 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:45.832838 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 417/832] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 13:54:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:45.834850 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329abe4-0042-4985-83a2-4177c67e8a03 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:45.835322 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329abe4-0042-4985-83a2-4177c67e8a03 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:45.835795 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329abe4-0042-4985-83a2-4177c67e8a03 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:45.844974 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 418/833] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:45.844974 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329abe4-0042-4985-83a2-4177c67e8a03 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:45.858625 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:45.893253 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98377fe4-3a84-468f-9980-e45dfd11a4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:45.894134 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98377fe4-3a84-468f-9980-e45dfd11a4bf None admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:45.894484 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98377fe4-3a84-468f-9980-e45dfd11a4bf None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:45.894908 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98377fe4-3a84-468f-9980-e45dfd11a4bf None admin] PATH_INFO: `/v3/projects/36f063d8f65d494a89dbbd72c53417c8/users/e07e83c0bdba47d39c09308eba1bdaae/roles/d17972f7fe97462b8f9f08aed9bc9cb5` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:45.914033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98377fe4-3a84-468f-9980-e45dfd11a4bf None admin] RBAC: Authorizing `identity:create_grant(project_id=36f063d8f65d494a89dbbd72c53417c8, user_id=e07e83c0bdba47d39c09308eba1bdaae, role_id=d17972f7fe97462b8f9f08aed9bc9cb5)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:45.915628 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98377fe4-3a84-468f-9980-e45dfd11a4bf None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:45.933256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 416/834] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 13:54:45 2022] PUT /identity/v3/projects/36f063d8f65d494a89dbbd72c53417c8/users/e07e83c0bdba47d39c09308eba1bdaae/roles/d17972f7fe97462b8f9f08aed9bc9cb5 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:54:45.934047 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98377fe4-3a84-468f-9980-e45dfd11a4bf None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:46.070201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329abe4-0042-4985-83a2-4177c67e8a03 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:46.104757 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:46.128880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98377fe4-3a84-468f-9980-e45dfd11a4bf None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:46.146929 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0554bba-4137-4b0d-b089-7d43e7be44ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:46.147675 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0554bba-4137-4b0d-b089-7d43e7be44ff None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:46.147980 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0554bba-4137-4b0d-b089-7d43e7be44ff None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:46.148285 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0554bba-4137-4b0d-b089-7d43e7be44ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:46.160986 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0554bba-4137-4b0d-b089-7d43e7be44ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:46.162559 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0554bba-4137-4b0d-b089-7d43e7be44ff None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:46.190100 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-a0554bba-4137-4b0d-b089-7d43e7be44ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:46.190100 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:46.190100 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:46.190100 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:46.190100 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:46.190100 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:46.190100 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:46.190100 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:46.190100 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:46.190100 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:46.190100 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:46.190100 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:46.190100 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:54:46.190100 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:54:46.190100 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:54:46.191729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:54:46.191729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:46.191729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:46.191729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:46.191729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:46.191729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:46.191729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:46.191729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:46.191729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:46.191729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:46.191729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:46.191729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:46.191729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:46.191729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:46.191729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:46.191729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:46.191729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:46.194181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:54:46.194181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:54:46.194181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:54:46.194181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:54:46.194181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:54:46.194181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:54:46.194181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:46.194181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:46.194181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 419/835] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:54:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:46.194181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d63ea5-3f67-4e9b-9fe1-28fffb7b58fd None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:46.195202 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d63ea5-3f67-4e9b-9fe1-28fffb7b58fd None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:46.195202 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d63ea5-3f67-4e9b-9fe1-28fffb7b58fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:46.202713 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be814dce-d112-4583-b3f9-da8d4b66858f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:46.204950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be814dce-d112-4583-b3f9-da8d4b66858f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:46.207341 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be814dce-d112-4583-b3f9-da8d4b66858f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:46.208194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be814dce-d112-4583-b3f9-da8d4b66858f None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:46.209821 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be814dce-d112-4583-b3f9-da8d4b66858f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:46.210629 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be814dce-d112-4583-b3f9-da8d4b66858f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:46.213481 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be814dce-d112-4583-b3f9-da8d4b66858f None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:46.221025 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-69d63ea5-3f67-4e9b-9fe1-28fffb7b58fd None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:46.222697 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-69d63ea5-3f67-4e9b-9fe1-28fffb7b58fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:46.226011 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 417/836] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 13:54:45 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:46.226011 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be814dce-d112-4583-b3f9-da8d4b66858f None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:46.282323 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:46.356741 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 420/837] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:54:46.380081 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d63ea5-3f67-4e9b-9fe1-28fffb7b58fd None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:46.433814 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be814dce-d112-4583-b3f9-da8d4b66858f None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:46.502375 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fecafe0-c42a-416e-b3a4-3a9be34c78f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:46.502683 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fecafe0-c42a-416e-b3a4-3a9be34c78f4 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:46.503101 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fecafe0-c42a-416e-b3a4-3a9be34c78f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:46.503461 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fecafe0-c42a-416e-b3a4-3a9be34c78f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:46.504404 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fecafe0-c42a-416e-b3a4-3a9be34c78f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:46.504788 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fecafe0-c42a-416e-b3a4-3a9be34c78f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:46.506449 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fecafe0-c42a-416e-b3a4-3a9be34c78f4 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:46.544118 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 418/838] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:54:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:46.553845 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fecafe0-c42a-416e-b3a4-3a9be34c78f4 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:46.598519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d63ea5-3f67-4e9b-9fe1-28fffb7b58fd None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:46.602080 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:46.655630 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95880bfc-703e-4b61-8528-ae7c091872bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:46.656407 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95880bfc-703e-4b61-8528-ae7c091872bf None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:46.656737 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95880bfc-703e-4b61-8528-ae7c091872bf None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:46.656923 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95880bfc-703e-4b61-8528-ae7c091872bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:46.674058 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95880bfc-703e-4b61-8528-ae7c091872bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:54:46.679806 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-95880bfc-703e-4b61-8528-ae7c091872bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:54:46.679806 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:46.679806 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:46.679806 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:46.679806 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:46.679806 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:46.679806 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:46.679806 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:46.679806 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:46.679806 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:46.679806 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:46.679806 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:46.679806 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:54:46.679806 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:54:46.679806 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:54:46.679806 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:54:46.681747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:54:46.681747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:54:46.681747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:54:46.681747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:54:46.681747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:46.681747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:46.681747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:46.681747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:46.681747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:46.681747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:46.681747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:46.681747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:46.681747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:46.681747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:46.681747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:46.681747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:46.683411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:46.683411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:46.683411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:46.683411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:46.683411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:54:46.683411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:54:46.683411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:54:46.683411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:54:46.683411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:54:46.683411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:54:46.683411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:54:46.683411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:46.689263 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 421/839] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:54:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:46.690422 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95880bfc-703e-4b61-8528-ae7c091872bf None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:46.740324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:46.773427 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fecafe0-c42a-416e-b3a4-3a9be34c78f4 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:46.854912 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7df0cf-a662-4aec-92ed-c1dfcb2afc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:46.856002 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7df0cf-a662-4aec-92ed-c1dfcb2afc9c None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:46.856567 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7df0cf-a662-4aec-92ed-c1dfcb2afc9c None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:46.857042 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7df0cf-a662-4aec-92ed-c1dfcb2afc9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:46.858593 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7df0cf-a662-4aec-92ed-c1dfcb2afc9c None admin] RBAC: Adding query filter params (role.id=3e1a9ff016c840c5b51ef7e3da9900ac, scope.project.id=36f063d8f65d494a89dbbd72c53417c8, user.id=5b3adc14dcaf4ac79b43bb97fa1ba1d4) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:46.859215 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7df0cf-a662-4aec-92ed-c1dfcb2afc9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:46.862482 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7df0cf-a662-4aec-92ed-c1dfcb2afc9c None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:46.880390 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 419/840] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:54:46 2022] GET /identity/v3/role_assignments?user.id=5b3adc14dcaf4ac79b43bb97fa1ba1d4&scope.project.id=36f063d8f65d494a89dbbd72c53417c8&role.id=3e1a9ff016c840c5b51ef7e3da9900ac => generated 608 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:54:46.929736 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95880bfc-703e-4b61-8528-ae7c091872bf None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:46.998210 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805d1dd-f7f3-4127-8a59-4f8d178998c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:46.999163 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805d1dd-f7f3-4127-8a59-4f8d178998c4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:46.999580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805d1dd-f7f3-4127-8a59-4f8d178998c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:47.000096 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805d1dd-f7f3-4127-8a59-4f8d178998c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:47.001284 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e805d1dd-f7f3-4127-8a59-4f8d178998c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:47.001756 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e805d1dd-f7f3-4127-8a59-4f8d178998c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:47.004192 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e805d1dd-f7f3-4127-8a59-4f8d178998c4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:47.020379 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 422/841] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:54:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:47.029250 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7df0cf-a662-4aec-92ed-c1dfcb2afc9c None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:47.235366 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7df0cf-a662-4aec-92ed-c1dfcb2afc9c None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:47.314089 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48064200-c48f-4cf5-84db-043881f72896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:47.315247 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48064200-c48f-4cf5-84db-043881f72896 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:47.315740 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48064200-c48f-4cf5-84db-043881f72896 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:47.316222 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48064200-c48f-4cf5-84db-043881f72896 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:47.339703 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48064200-c48f-4cf5-84db-043881f72896 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:47.342418 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48064200-c48f-4cf5-84db-043881f72896 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:47.372757 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-48064200-c48f-4cf5-84db-043881f72896 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:47.372757 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:47.372757 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:47.372757 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:47.372757 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:47.372757 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:47.372757 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:47.372757 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:47.372757 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:47.372757 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:47.372757 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:47.372757 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:47.372757 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:47.372757 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:47.372757 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:47.374487 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:47.374487 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:47.374487 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:47.374487 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:47.374487 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:47.374487 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:47.374487 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:47.374487 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:47.374487 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:47.374487 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:47.374487 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:47.374487 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:47.374487 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:47.374487 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:47.374487 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:47.374487 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:47.374487 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:47.376234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:47.376234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:47.376234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:47.376234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:47.380335 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 420/842] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:47.388973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805d1dd-f7f3-4127-8a59-4f8d178998c4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:47.443233 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:47.599344 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805d1dd-f7f3-4127-8a59-4f8d178998c4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:47.646868 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c9b1e-c1e4-4d69-965f-41808e60687e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:47.647957 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c9b1e-c1e4-4d69-965f-41808e60687e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:47.648373 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c9b1e-c1e4-4d69-965f-41808e60687e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:47.648727 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c9b1e-c1e4-4d69-965f-41808e60687e None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:47.650009 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41c9b1e-c1e4-4d69-965f-41808e60687e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:47.650634 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41c9b1e-c1e4-4d69-965f-41808e60687e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:47.653111 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41c9b1e-c1e4-4d69-965f-41808e60687e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:47.672862 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 423/843] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:47.681649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48064200-c48f-4cf5-84db-043881f72896 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:47.690282 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:47.893302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48064200-c48f-4cf5-84db-043881f72896 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:47.963644 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b42d557-b020-403e-9782-8c781db5c286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:47.965132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b42d557-b020-403e-9782-8c781db5c286 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:47.965824 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b42d557-b020-403e-9782-8c781db5c286 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:47.966515 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b42d557-b020-403e-9782-8c781db5c286 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:47.983059 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b42d557-b020-403e-9782-8c781db5c286 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:47.986623 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b42d557-b020-403e-9782-8c781db5c286 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:48.011137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-2b42d557-b020-403e-9782-8c781db5c286 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 13:54:48.011137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:48.011137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:48.011137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:48.011137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:48.011137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:48.011137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:48.011137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:48.011137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:48.011137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:48.011137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:48.011137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:48.011137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:54:48.011137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:54:48.011137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:54:48.011137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:54:48.013494 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:48.013494 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:48.013494 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:54:48.013494 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:48.013494 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:54:48.013494 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:48.013494 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:48.013494 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:48.013494 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:48.013494 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:48.013494 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:48.013494 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:48.013494 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:48.013494 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:48.013494 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:48.013494 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:48.013494 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:48.013494 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:48.015717 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:48.015717 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:48.015717 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:54:48.015717 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:54:48.015717 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:54:48.015717 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:54:48.015717 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:54:48.015717 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:54:48.015717 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 13:54:48.015717 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:48.020026 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 421/844] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:54:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:48.029810 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c9b1e-c1e4-4d69-965f-41808e60687e None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:48.197730 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:48.228178 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c9b1e-c1e4-4d69-965f-41808e60687e None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:48.294541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3880226-1d33-45bf-801b-97f3fb45dbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:48.295551 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3880226-1d33-45bf-801b-97f3fb45dbe9 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:48.296002 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3880226-1d33-45bf-801b-97f3fb45dbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:48.296538 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3880226-1d33-45bf-801b-97f3fb45dbe9 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:48.298144 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3880226-1d33-45bf-801b-97f3fb45dbe9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:48.298656 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3880226-1d33-45bf-801b-97f3fb45dbe9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:48.301407 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3880226-1d33-45bf-801b-97f3fb45dbe9 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:48.330413 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 424/845] 10.222.0.45 () {56 vars in 1135 bytes} [Mon Aug 1 13:54:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:48.339883 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b42d557-b020-403e-9782-8c781db5c286 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:48.478941 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:48.570063 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b42d557-b020-403e-9782-8c781db5c286 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:48.638666 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6266ced5-59ba-42e4-8534-835c903fd35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:48.639742 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6266ced5-59ba-42e4-8534-835c903fd35d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:48.640224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6266ced5-59ba-42e4-8534-835c903fd35d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:48.640893 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6266ced5-59ba-42e4-8534-835c903fd35d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:48.786092 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6266ced5-59ba-42e4-8534-835c903fd35d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:48.788538 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6266ced5-59ba-42e4-8534-835c903fd35d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:48.916552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:48.993549 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 422/846] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:54:48 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 655 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:54:49.002289 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3880226-1d33-45bf-801b-97f3fb45dbe9 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:49.060085 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b312e3-118a-44ac-b6ee-ab03945ec9a1 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:49.060442 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b312e3-118a-44ac-b6ee-ab03945ec9a1 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:49.060864 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b312e3-118a-44ac-b6ee-ab03945ec9a1 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:49.063853 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 423/847] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:49.073136 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ac181-dddc-4f29-bdc0-ac5339257305 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:49.078484 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ac181-dddc-4f29-bdc0-ac5339257305 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:49.078484 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ac181-dddc-4f29-bdc0-ac5339257305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:49.115031 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-237ac181-dddc-4f29-bdc0-ac5339257305 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:49.118379 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-237ac181-dddc-4f29-bdc0-ac5339257305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:49.164617 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3880226-1d33-45bf-801b-97f3fb45dbe9 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:49.222720 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ef931-2e30-4973-ab03-319160d9b28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:49.223402 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ef931-2e30-4973-ab03-319160d9b28b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:49.223685 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ef931-2e30-4973-ab03-319160d9b28b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:49.223960 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ef931-2e30-4973-ab03-319160d9b28b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:49.236964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ef931-2e30-4973-ab03-319160d9b28b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:49.238515 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ef931-2e30-4973-ab03-319160d9b28b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:49.238781 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 424/848] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:49 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:54:49.247590 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:49.269137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ac181-dddc-4f29-bdc0-ac5339257305 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:49.272102 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-832ef931-2e30-4973-ab03-319160d9b28b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:49.272102 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:49.272102 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:49.272102 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:49.272102 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:49.272102 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:49.272102 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:49.272102 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:49.272102 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:49.272102 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:49.272102 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:49.272102 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:49.272102 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:49.272102 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:49.272102 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:49.273041 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:49.273041 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:49.273041 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:49.273041 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:49.273041 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:49.273041 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:49.273041 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:49.273041 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:49.273041 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:49.273041 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:49.273041 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:49.273041 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:49.273041 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:49.273041 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:49.273041 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:49.273041 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:49.273041 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:49.273950 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:49.273950 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:49.273950 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:49.273950 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:49.276546 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 425/849] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:54:49.285939 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ef931-2e30-4973-ab03-319160d9b28b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:49.460434 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:49.471736 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ac181-dddc-4f29-bdc0-ac5339257305 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:49.491558 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ef931-2e30-4973-ab03-319160d9b28b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:49.530501 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aafac9d-e115-4173-9fd7-6e6d87c9a84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:49.531061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafac9d-e115-4173-9fd7-6e6d87c9a84e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:49.531348 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafac9d-e115-4173-9fd7-6e6d87c9a84e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:49.531621 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafac9d-e115-4173-9fd7-6e6d87c9a84e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:49.550276 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aafac9d-e115-4173-9fd7-6e6d87c9a84e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:54:49.556751 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-9aafac9d-e115-4173-9fd7-6e6d87c9a84e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:54:49.556751 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:49.556751 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:49.556751 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:49.556751 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:49.556751 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:49.556751 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:49.556751 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:49.556751 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:49.556751 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:49.556751 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:49.556751 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:49.556751 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:54:49.556751 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:54:49.556751 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:54:49.556751 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:54:49.558545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:54:49.558545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:54:49.558545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:54:49.558545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:54:49.558545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:49.558545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:49.558545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:49.558545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:49.558545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:49.558545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:49.558545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:49.558545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:49.558545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:49.558545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:49.558545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:49.558545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:49.560160 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:49.560160 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:49.560160 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:49.560160 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:49.560160 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:54:49.560160 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:54:49.560160 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:54:49.560160 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:54:49.560160 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:54:49.560160 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:54:49.560160 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:54:49.560160 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:49.560160 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 425/850] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 13:54:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:49.567448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168dbc80-58b0-44a3-b27f-6a4a622bb7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:49.567448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168dbc80-58b0-44a3-b27f-6a4a622bb7df None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:49.567448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168dbc80-58b0-44a3-b27f-6a4a622bb7df None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:49.567448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168dbc80-58b0-44a3-b27f-6a4a622bb7df None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:49.568514 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168dbc80-58b0-44a3-b27f-6a4a622bb7df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:49.568514 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168dbc80-58b0-44a3-b27f-6a4a622bb7df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:49.568966 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aafac9d-e115-4173-9fd7-6e6d87c9a84e None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:49.569564 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168dbc80-58b0-44a3-b27f-6a4a622bb7df None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:49.588381 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 426/851] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:49.594047 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168dbc80-58b0-44a3-b27f-6a4a622bb7df None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:49.686038 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:49.734235 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aafac9d-e115-4173-9fd7-6e6d87c9a84e None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:49.782121 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcbc23d-944f-421f-a2b3-32e8bbd6cef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:49.783188 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcbc23d-944f-421f-a2b3-32e8bbd6cef9 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:49.783477 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcbc23d-944f-421f-a2b3-32e8bbd6cef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:49.783862 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcbc23d-944f-421f-a2b3-32e8bbd6cef9 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:49.784603 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcbc23d-944f-421f-a2b3-32e8bbd6cef9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:49.785005 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcbc23d-944f-421f-a2b3-32e8bbd6cef9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:49.787435 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcbc23d-944f-421f-a2b3-32e8bbd6cef9 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:49.797226 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168dbc80-58b0-44a3-b27f-6a4a622bb7df None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:49.803911 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 426/852] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 13:54:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:54:49.803911 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceec51d-d90c-4ce9-9592-4dbdc2806f09 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:49.803911 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceec51d-d90c-4ce9-9592-4dbdc2806f09 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:49.803911 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceec51d-d90c-4ce9-9592-4dbdc2806f09 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:49.808681 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 427/853] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:49.810445 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceec51d-d90c-4ce9-9592-4dbdc2806f09 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:49.844260 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:49.859529 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a0283-2e3e-4898-8e98-0323317c361c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:49.861354 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a0283-2e3e-4898-8e98-0323317c361c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:49.861885 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a0283-2e3e-4898-8e98-0323317c361c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:49.862285 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a0283-2e3e-4898-8e98-0323317c361c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:49.874599 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a0283-2e3e-4898-8e98-0323317c361c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:49.876560 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a0283-2e3e-4898-8e98-0323317c361c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:49.909039 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-c24a0283-2e3e-4898-8e98-0323317c361c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:49.909039 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:49.909039 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:49.909039 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:49.909039 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:49.909039 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:49.909039 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:49.909039 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:49.909039 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:49.909039 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:49.909039 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:49.909039 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:49.909039 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:54:49.909039 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:54:49.909039 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:54:49.910857 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:54:49.910857 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:49.910857 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:49.910857 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:49.910857 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:49.910857 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:49.910857 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:49.910857 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:49.910857 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:49.910857 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:49.910857 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:49.910857 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:49.910857 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:49.910857 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:49.910857 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:49.910857 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:49.910857 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:49.912576 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:54:49.912576 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:54:49.912576 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:54:49.912576 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:54:49.912576 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:54:49.912576 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:54:49.912576 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:49.912576 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:49.912576 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 427/854] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:54:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:49.913524 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac420e-ae39-431b-b2f7-bfb21d7c2bbd None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:49.913664 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac420e-ae39-431b-b2f7-bfb21d7c2bbd None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:49.915131 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac420e-ae39-431b-b2f7-bfb21d7c2bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:49.944179 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-4fac420e-ae39-431b-b2f7-bfb21d7c2bbd None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:49.948596 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4fac420e-ae39-431b-b2f7-bfb21d7c2bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:50.034765 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceec51d-d90c-4ce9-9592-4dbdc2806f09 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:50.062269 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 428/855] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:49 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:54:50.063674 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac420e-ae39-431b-b2f7-bfb21d7c2bbd None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:50.091261 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfbae6e-de6e-4149-8ed3-99b1e2fef7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:50.092393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfbae6e-de6e-4149-8ed3-99b1e2fef7d2 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:50.092932 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfbae6e-de6e-4149-8ed3-99b1e2fef7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:50.096659 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfbae6e-de6e-4149-8ed3-99b1e2fef7d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:50.116074 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfbae6e-de6e-4149-8ed3-99b1e2fef7d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:50.117685 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfbae6e-de6e-4149-8ed3-99b1e2fef7d2 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:50.136172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-fbfbae6e-de6e-4149-8ed3-99b1e2fef7d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:50.136172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:50.136172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:50.136172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:50.136172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:50.136172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:50.136172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:50.136172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:50.136172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:50.136172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:50.136172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:50.136172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:50.136172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:50.136172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:50.136172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:50.137942 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:50.137942 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:50.137942 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:50.137942 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:50.137942 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:50.137942 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:50.137942 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:50.137942 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:50.137942 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:50.137942 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:50.137942 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:50.137942 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:50.137942 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:50.137942 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:50.137942 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:50.137942 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:50.137942 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:50.140095 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:50.140095 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:50.140095 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:50.140095 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:50.140095 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 428/856] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:50.140095 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5a073-a6fe-4385-8533-401ff8d65c50 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:50.140791 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5a073-a6fe-4385-8533-401ff8d65c50 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:50.140791 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5a073-a6fe-4385-8533-401ff8d65c50 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:50.148438 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 429/857] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:50.148438 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5a073-a6fe-4385-8533-401ff8d65c50 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:50.150234 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:50.221462 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac420e-ae39-431b-b2f7-bfb21d7c2bbd None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:50.272732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b52317-f24b-4740-9afa-87395d8d324d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:50.273689 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:50.273862 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b52317-f24b-4740-9afa-87395d8d324d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:50.274320 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b52317-f24b-4740-9afa-87395d8d324d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:50.274824 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b52317-f24b-4740-9afa-87395d8d324d None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:50.276395 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b52317-f24b-4740-9afa-87395d8d324d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:50.276989 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b52317-f24b-4740-9afa-87395d8d324d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:50.280656 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b52317-f24b-4740-9afa-87395d8d324d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:50.306703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 429/858] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:54:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:50.306703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b52317-f24b-4740-9afa-87395d8d324d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:50.338805 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5a073-a6fe-4385-8533-401ff8d65c50 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:50.389105 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d675e142-c8af-42de-a9de-a6259c94c5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:50.390132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d675e142-c8af-42de-a9de-a6259c94c5c6 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:50.390657 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d675e142-c8af-42de-a9de-a6259c94c5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:50.391130 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d675e142-c8af-42de-a9de-a6259c94c5c6 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:50.392484 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d675e142-c8af-42de-a9de-a6259c94c5c6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:50.395004 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d675e142-c8af-42de-a9de-a6259c94c5c6 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:50.416960 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 430/859] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Aug 1 13:54:50 2022] POST /identity/v3/services => generated 251 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:54:50.418570 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076284df-d291-456c-b15b-c1157f125125 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:50.418865 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076284df-d291-456c-b15b-c1157f125125 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:50.419174 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076284df-d291-456c-b15b-c1157f125125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:50.454592 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-076284df-d291-456c-b15b-c1157f125125 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:50.456510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b52317-f24b-4740-9afa-87395d8d324d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:50.457445 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-076284df-d291-456c-b15b-c1157f125125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:50.461413 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:50.525967 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170b5c0-6750-43de-87bc-00e2c64e021f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:50.526791 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170b5c0-6750-43de-87bc-00e2c64e021f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:50.527203 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170b5c0-6750-43de-87bc-00e2c64e021f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:50.527598 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170b5c0-6750-43de-87bc-00e2c64e021f None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:50.528580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0170b5c0-6750-43de-87bc-00e2c64e021f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:50.529033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0170b5c0-6750-43de-87bc-00e2c64e021f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:50.530604 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0170b5c0-6750-43de-87bc-00e2c64e021f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:50.558989 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 430/860] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:50.558989 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170b5c0-6750-43de-87bc-00e2c64e021f None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:50.580561 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 431/861] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:50 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:54:50.580561 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076284df-d291-456c-b15b-c1157f125125 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:50.690139 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:50.738802 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170b5c0-6750-43de-87bc-00e2c64e021f None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:50.742244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076284df-d291-456c-b15b-c1157f125125 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:50.789591 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a03040-d085-4912-8277-a09c8629e7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:50.790238 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a03040-d085-4912-8277-a09c8629e7bb None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:50.790983 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a03040-d085-4912-8277-a09c8629e7bb None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:50.790983 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a03040-d085-4912-8277-a09c8629e7bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:50.791600 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a03040-d085-4912-8277-a09c8629e7bb None admin] RBAC: Adding query filter params (role.id=3e1a9ff016c840c5b51ef7e3da9900ac, scope.project.id=36f063d8f65d494a89dbbd72c53417c8, user.id=0cda7f365ba0405daaf86870960f12d2) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:50.791907 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a03040-d085-4912-8277-a09c8629e7bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:50.793493 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a03040-d085-4912-8277-a09c8629e7bb None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:50.801010 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 431/862] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:54:50 2022] GET /identity/v3/role_assignments?user.id=0cda7f365ba0405daaf86870960f12d2&scope.project.id=36f063d8f65d494a89dbbd72c53417c8&role.id=3e1a9ff016c840c5b51ef7e3da9900ac => generated 608 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:50.802088 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a03040-d085-4912-8277-a09c8629e7bb None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:50.811767 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec8a65-107b-41de-84c0-c90ff3a80ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:50.813342 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec8a65-107b-41de-84c0-c90ff3a80ccf None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:50.813887 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec8a65-107b-41de-84c0-c90ff3a80ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:50.814441 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec8a65-107b-41de-84c0-c90ff3a80ccf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:50.831505 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec8a65-107b-41de-84c0-c90ff3a80ccf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:50.834923 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec8a65-107b-41de-84c0-c90ff3a80ccf None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:50.848596 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:50.867541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-cfec8a65-107b-41de-84c0-c90ff3a80ccf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 13:54:50.867541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:50.867541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:50.867541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:50.867541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:50.867541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:50.867541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:50.867541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:50.867541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:50.867541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:50.867541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:50.867541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:50.867541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:54:50.867541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:54:50.867541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:54:50.867541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:54:50.869344 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:50.869344 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:50.869344 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:54:50.869344 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:50.869344 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:54:50.869344 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:50.869344 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:50.869344 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:50.869344 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:50.869344 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:50.869344 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:50.869344 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:50.869344 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:50.869344 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:50.869344 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:50.869344 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:50.869344 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:50.869344 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:50.871649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:50.871649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:50.871649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:54:50.871649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:54:50.871649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:54:50.871649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:54:50.871649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:54:50.871649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:54:50.871649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 13:54:50.871649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:50.871649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 432/863] 10.222.0.45 () {56 vars in 1117 bytes} [Mon Aug 1 13:54:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:50.879043 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec8a65-107b-41de-84c0-c90ff3a80ccf None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:51.049756 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a03040-d085-4912-8277-a09c8629e7bb None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:51.093022 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec8a65-107b-41de-84c0-c90ff3a80ccf None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:51.126014 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:51.128039 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d5210-185e-4f85-ab75-1547601a73d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:51.129245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d5210-185e-4f85-ab75-1547601a73d8 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:51.129833 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d5210-185e-4f85-ab75-1547601a73d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:51.130373 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d5210-185e-4f85-ab75-1547601a73d8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:51.131278 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d5210-185e-4f85-ab75-1547601a73d8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:51.133460 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d5210-185e-4f85-ab75-1547601a73d8 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:51.150648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-b69d5210-185e-4f85-ab75-1547601a73d8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 13:54:51.150648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:51.150648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:54:51.150648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:54:51.150648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:54:51.150648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:54:51.150648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:54:51.150648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:54:51.150648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 13:54:51.150648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:51.150648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:54:51.150648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:51.150648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:51.150648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:51.150648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:51.150648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:51.152399 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:51.152399 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:51.152399 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:51.152399 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:51.152399 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:51.152399 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:51.152399 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:51.152399 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:54:51.152399 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:54:51.152399 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:54:51.152399 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:54:51.152399 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:51.152399 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:51.152399 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:51.152399 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:51.152399 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:51.154284 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:51.154284 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:51.154284 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:51.154284 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:51.154284 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:51.154284 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:51.154284 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:51.154284 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:51.154284 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:51.154284 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:51.154284 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:51.154284 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:54:51.154284 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:54:51.154284 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 13:54:51.154284 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:51.154284 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 432/864] 10.222.0.45 () {56 vars in 1132 bytes} [Mon Aug 1 13:54:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:51.163550 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d5210-185e-4f85-ab75-1547601a73d8 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:51.171672 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba82162-1e93-4744-9b7d-26bb0535e11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:51.173037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba82162-1e93-4744-9b7d-26bb0535e11f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:51.173521 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba82162-1e93-4744-9b7d-26bb0535e11f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:51.173993 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba82162-1e93-4744-9b7d-26bb0535e11f None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:51.175439 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba82162-1e93-4744-9b7d-26bb0535e11f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:51.175754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba82162-1e93-4744-9b7d-26bb0535e11f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:51.177367 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba82162-1e93-4744-9b7d-26bb0535e11f None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:51.215399 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 433/865] 10.222.0.45 () {56 vars in 1149 bytes} [Mon Aug 1 13:54:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:51.216949 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba82162-1e93-4744-9b7d-26bb0535e11f None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:51.340984 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:51.365028 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba82162-1e93-4744-9b7d-26bb0535e11f None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:51.368018 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d5210-185e-4f85-ab75-1547601a73d8 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:51.407082 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6675d0-7e7e-46ab-b476-1bdbb7ccb5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:51.407715 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6675d0-7e7e-46ab-b476-1bdbb7ccb5e6 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:51.407982 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6675d0-7e7e-46ab-b476-1bdbb7ccb5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:51.408258 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6675d0-7e7e-46ab-b476-1bdbb7ccb5e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:51.442905 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861587a-bcf3-4bf4-ab7e-892e7abee0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:51.444490 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861587a-bcf3-4bf4-ab7e-892e7abee0f1 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:51.445336 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861587a-bcf3-4bf4-ab7e-892e7abee0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:51.446288 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861587a-bcf3-4bf4-ab7e-892e7abee0f1 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:51.447763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1861587a-bcf3-4bf4-ab7e-892e7abee0f1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:51.448550 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1861587a-bcf3-4bf4-ab7e-892e7abee0f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:51.451581 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1861587a-bcf3-4bf4-ab7e-892e7abee0f1 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:51.460776 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 433/866] 10.222.0.45 () {56 vars in 1126 bytes} [Mon Aug 1 13:54:51 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:51.469606 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861587a-bcf3-4bf4-ab7e-892e7abee0f1 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:51.491399 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:51.532630 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6675d0-7e7e-46ab-b476-1bdbb7ccb5e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:51.537864 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6675d0-7e7e-46ab-b476-1bdbb7ccb5e6 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:51.610826 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861587a-bcf3-4bf4-ab7e-892e7abee0f1 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:51.658404 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 434/867] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:54:51 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:54:51.661811 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6675d0-7e7e-46ab-b476-1bdbb7ccb5e6 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:51.664850 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e55f96-79c0-41a7-b5fb-a07d94af4d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:51.666683 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e55f96-79c0-41a7-b5fb-a07d94af4d66 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:51.666683 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e55f96-79c0-41a7-b5fb-a07d94af4d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:51.667188 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e55f96-79c0-41a7-b5fb-a07d94af4d66 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:51.668334 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e55f96-79c0-41a7-b5fb-a07d94af4d66 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:51.668885 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e55f96-79c0-41a7-b5fb-a07d94af4d66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:51.671447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e55f96-79c0-41a7-b5fb-a07d94af4d66 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:51.689225 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 434/868] 10.222.0.45 () {56 vars in 1126 bytes} [Mon Aug 1 13:54:51 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:51.762259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:51.794505 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6675d0-7e7e-46ab-b476-1bdbb7ccb5e6 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:51.840648 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb7d67-7302-4371-9bff-e7b31997d4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:51.841680 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb7d67-7302-4371-9bff-e7b31997d4b9 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:51.841680 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb7d67-7302-4371-9bff-e7b31997d4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:51.841951 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb7d67-7302-4371-9bff-e7b31997d4b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:51.853839 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb7d67-7302-4371-9bff-e7b31997d4b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:51.855367 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb7d67-7302-4371-9bff-e7b31997d4b9 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:51.873381 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-b1cb7d67-7302-4371-9bff-e7b31997d4b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:51.873381 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:51.873381 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:51.873381 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:51.873381 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:51.873381 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:51.873381 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:51.873381 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:51.873381 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:51.873381 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:51.873381 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:51.873381 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:51.873381 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:51.873381 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:51.873381 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:51.874395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:51.874395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:51.874395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:51.874395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:51.874395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:51.874395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:51.874395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:51.874395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:51.874395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:51.874395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:51.874395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:51.874395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:51.874395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:51.874395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:51.874395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:51.874395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:51.874395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:51.875865 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:51.875865 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:51.875865 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:51.875865 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:51.875865 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 435/869] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:51.881023 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e55f96-79c0-41a7-b5fb-a07d94af4d66 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:51.917868 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:52.072048 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e55f96-79c0-41a7-b5fb-a07d94af4d66 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:52.133855 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99389ffb-2340-4bb0-8667-fa454465678a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:52.134813 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99389ffb-2340-4bb0-8667-fa454465678a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:52.135265 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99389ffb-2340-4bb0-8667-fa454465678a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:52.135969 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99389ffb-2340-4bb0-8667-fa454465678a None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:52.137143 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99389ffb-2340-4bb0-8667-fa454465678a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:52.137642 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99389ffb-2340-4bb0-8667-fa454465678a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:52.140006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99389ffb-2340-4bb0-8667-fa454465678a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:52.163397 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 435/870] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:52.172465 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb7d67-7302-4371-9bff-e7b31997d4b9 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:52.394165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb7d67-7302-4371-9bff-e7b31997d4b9 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:52.467192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be433d-69af-48ed-8583-99904aa9b4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:52.468221 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be433d-69af-48ed-8583-99904aa9b4b4 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:52.469367 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be433d-69af-48ed-8583-99904aa9b4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:52.469934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be433d-69af-48ed-8583-99904aa9b4b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:52.471137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:52.482560 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04be433d-69af-48ed-8583-99904aa9b4b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:52.484482 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04be433d-69af-48ed-8583-99904aa9b4b4 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:52.504064 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-04be433d-69af-48ed-8583-99904aa9b4b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:52.504064 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:52.504064 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:52.504064 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:52.504064 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:52.504064 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:52.504064 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:52.504064 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:52.504064 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:52.504064 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:52.504064 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:52.504064 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:52.504064 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:54:52.504064 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:54:52.504064 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:54:52.505340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:54:52.505340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:52.505340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:52.505340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:52.505340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:52.505340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:52.505340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:52.505340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:52.505340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:52.505340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:52.505340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:52.505340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:52.505340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:52.505340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:52.505340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:52.505340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:52.505340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:52.506647 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:54:52.506647 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:54:52.506647 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:54:52.506647 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:54:52.506647 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:54:52.506647 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:54:52.506647 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:54:52.506647 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:52.506647 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 436/871] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:54:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:52.511988 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99389ffb-2340-4bb0-8667-fa454465678a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:52.636787 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:52.668511 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99389ffb-2340-4bb0-8667-fa454465678a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:52.714438 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cb31c6-b596-4802-8ed8-6c42963668b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:52.715200 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb31c6-b596-4802-8ed8-6c42963668b6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:52.715602 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb31c6-b596-4802-8ed8-6c42963668b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:52.715946 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb31c6-b596-4802-8ed8-6c42963668b6 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:52.717266 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cb31c6-b596-4802-8ed8-6c42963668b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:52.717688 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cb31c6-b596-4802-8ed8-6c42963668b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:52.719646 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cb31c6-b596-4802-8ed8-6c42963668b6 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:52.740760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 436/872] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:54:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:52.743498 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be433d-69af-48ed-8583-99904aa9b4b4 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:52.889072 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e82088-1aa0-473d-81c6-37f242bc899f None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:52.889917 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e82088-1aa0-473d-81c6-37f242bc899f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:52.890497 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e82088-1aa0-473d-81c6-37f242bc899f None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:52.893525 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 437/873] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:54:52.900501 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99c361-8652-41cb-8006-ba4777843113 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:52.901110 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99c361-8652-41cb-8006-ba4777843113 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:52.901612 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99c361-8652-41cb-8006-ba4777843113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:52.923314 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-3b99c361-8652-41cb-8006-ba4777843113 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:52.925026 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3b99c361-8652-41cb-8006-ba4777843113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:52.947422 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be433d-69af-48ed-8583-99904aa9b4b4 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:52.990812 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8706b-ae73-4df1-b5a1-ac91a03a06e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:52.991159 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8706b-ae73-4df1-b5a1-ac91a03a06e1 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:52.991379 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8706b-ae73-4df1-b5a1-ac91a03a06e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:52.991573 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8706b-ae73-4df1-b5a1-ac91a03a06e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:52.993568 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8706b-ae73-4df1-b5a1-ac91a03a06e1 None admin] RBAC: Adding query filter params (role.id=d17972f7fe97462b8f9f08aed9bc9cb5, scope.project.id=36f063d8f65d494a89dbbd72c53417c8, user.id=e07e83c0bdba47d39c09308eba1bdaae) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:52.993568 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8706b-ae73-4df1-b5a1-ac91a03a06e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:52.994954 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8706b-ae73-4df1-b5a1-ac91a03a06e1 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:53.012306 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 437/874] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:54:52 2022] GET /identity/v3/role_assignments?user.id=e07e83c0bdba47d39c09308eba1bdaae&scope.project.id=36f063d8f65d494a89dbbd72c53417c8&role.id=d17972f7fe97462b8f9f08aed9bc9cb5 => generated 608 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:54:53.031942 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 438/875] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:52 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:54:53.055215 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8706b-ae73-4df1-b5a1-ac91a03a06e1 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:53.156305 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:53.261147 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8706b-ae73-4df1-b5a1-ac91a03a06e1 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:53.315879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa20f032-952e-4d5a-9f96-3e34253792cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:53.317051 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa20f032-952e-4d5a-9f96-3e34253792cf None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:53.317516 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa20f032-952e-4d5a-9f96-3e34253792cf None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:53.317947 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa20f032-952e-4d5a-9f96-3e34253792cf None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:53.318759 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa20f032-952e-4d5a-9f96-3e34253792cf None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:53.321114 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa20f032-952e-4d5a-9f96-3e34253792cf None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:53.335977 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-fa20f032-952e-4d5a-9f96-3e34253792cf None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 13:54:53.335977 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:53.335977 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:54:53.335977 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:54:53.335977 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:54:53.335977 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:54:53.335977 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:54:53.335977 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:54:53.335977 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 13:54:53.335977 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:53.335977 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:54:53.335977 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:53.335977 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:53.335977 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:53.335977 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:53.335977 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:53.335977 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:53.337900 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:53.337900 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:53.337900 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:53.337900 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:53.337900 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:53.337900 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:53.337900 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:54:53.337900 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:54:53.337900 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:54:53.337900 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:54:53.337900 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:53.337900 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:53.337900 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:53.337900 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:53.337900 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:53.337900 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:53.339886 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:53.339886 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:53.339886 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:53.339886 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:53.339886 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:53.339886 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:53.339886 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:53.339886 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:53.339886 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:53.339886 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:53.339886 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:54:53.339886 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:54:53.339886 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 13:54:53.339886 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:53.339886 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 438/876] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 13:54:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:53.346933 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b99c361-8652-41cb-8006-ba4777843113 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:53.449955 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:53.509036 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b99c361-8652-41cb-8006-ba4777843113 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:53.574978 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36c27a-f89a-4d03-bd52-3b241a7483fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:53.576013 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36c27a-f89a-4d03-bd52-3b241a7483fd None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:53.576767 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36c27a-f89a-4d03-bd52-3b241a7483fd None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:53.577261 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36c27a-f89a-4d03-bd52-3b241a7483fd None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:53.578582 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af36c27a-f89a-4d03-bd52-3b241a7483fd None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:53.579124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af36c27a-f89a-4d03-bd52-3b241a7483fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:53.581733 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af36c27a-f89a-4d03-bd52-3b241a7483fd None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:53.591486 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 439/877] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 13:54:53 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:53.600831 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa20f032-952e-4d5a-9f96-3e34253792cf None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:53.799844 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa20f032-952e-4d5a-9f96-3e34253792cf None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:53.842014 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:53.868353 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064ead3-ef16-40aa-b709-d43d6c6b77b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:53.869444 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064ead3-ef16-40aa-b709-d43d6c6b77b7 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:53.869944 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064ead3-ef16-40aa-b709-d43d6c6b77b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:53.870436 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064ead3-ef16-40aa-b709-d43d6c6b77b7 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:53.871744 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0064ead3-ef16-40aa-b709-d43d6c6b77b7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:53.872308 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0064ead3-ef16-40aa-b709-d43d6c6b77b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:53.874990 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0064ead3-ef16-40aa-b709-d43d6c6b77b7 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:53.885275 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 439/878] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 13:54:53 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:53.895096 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36c27a-f89a-4d03-bd52-3b241a7483fd None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:54.049004 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c195eb4-c414-4e23-962a-54c350c3e4aa None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:54.050024 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c195eb4-c414-4e23-962a-54c350c3e4aa None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:54.050501 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c195eb4-c414-4e23-962a-54c350c3e4aa None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:54.052394 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 440/879] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:54:54.061364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789890a-81a3-4bbd-a2b8-bf5830b385d4 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:54.061863 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789890a-81a3-4bbd-a2b8-bf5830b385d4 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:54.062388 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789890a-81a3-4bbd-a2b8-bf5830b385d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:54.085729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-3789890a-81a3-4bbd-a2b8-bf5830b385d4 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:54.087372 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3789890a-81a3-4bbd-a2b8-bf5830b385d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:54.096168 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36c27a-f89a-4d03-bd52-3b241a7483fd None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:54.140908 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:54.151807 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479d2ec-8ef8-4e85-bdfc-9ce3a8aca5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:54.153313 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479d2ec-8ef8-4e85-bdfc-9ce3a8aca5d0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:54.154074 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479d2ec-8ef8-4e85-bdfc-9ce3a8aca5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:54.155269 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479d2ec-8ef8-4e85-bdfc-9ce3a8aca5d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:54.158524 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1479d2ec-8ef8-4e85-bdfc-9ce3a8aca5d0 None admin] RBAC: Adding query filter params (interface=public, service_id=333cfc401ecb44f09cb3fd5b4832786d, region_id=RegionOne) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:54.159409 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1479d2ec-8ef8-4e85-bdfc-9ce3a8aca5d0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:54.162269 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1479d2ec-8ef8-4e85-bdfc-9ce3a8aca5d0 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:54.181130 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 440/880] 10.222.0.45 () {56 vars in 1251 bytes} [Mon Aug 1 13:54:53 2022] GET /identity/v3/endpoints?service_id=333cfc401ecb44f09cb3fd5b4832786d&interface=public®ion_id=RegionOne => generated 194 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:54.182665 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479d2ec-8ef8-4e85-bdfc-9ce3a8aca5d0 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:54.227414 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 441/881] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:54 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:54:54.263018 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3789890a-81a3-4bbd-a2b8-bf5830b385d4 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:54.331994 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479d2ec-8ef8-4e85-bdfc-9ce3a8aca5d0 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:54.376978 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771cf50b-3b32-48f2-9b62-32a160d53933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:54.378336 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771cf50b-3b32-48f2-9b62-32a160d53933 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:54.378859 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771cf50b-3b32-48f2-9b62-32a160d53933 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:54.379587 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771cf50b-3b32-48f2-9b62-32a160d53933 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:54.381028 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771cf50b-3b32-48f2-9b62-32a160d53933 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:54.383555 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771cf50b-3b32-48f2-9b62-32a160d53933 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:54.400693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 441/882] 10.222.0.45 () {56 vars in 1087 bytes} [Mon Aug 1 13:54:54 2022] GET /identity/v3/services => generated 1011 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 13:54:54.462023 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3789890a-81a3-4bbd-a2b8-bf5830b385d4 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:54.491080 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:54.525281 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c6b48-8163-490f-aef6-48e54dce82e7 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:54.525781 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c6b48-8163-490f-aef6-48e54dce82e7 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:54.526838 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c6b48-8163-490f-aef6-48e54dce82e7 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:54.536475 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78193b2c-a4b0-42af-bb4d-67f68808fcca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:54.536475 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78193b2c-a4b0-42af-bb4d-67f68808fcca None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:54.536475 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78193b2c-a4b0-42af-bb4d-67f68808fcca None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:54.537028 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78193b2c-a4b0-42af-bb4d-67f68808fcca None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:54.537158 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 442/883] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:54 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:54:54.538677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78193b2c-a4b0-42af-bb4d-67f68808fcca None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:54.539372 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9d28c-5c28-4c6a-a2e3-17e897af4eac None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:54.539881 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9d28c-5c28-4c6a-a2e3-17e897af4eac None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:54.544314 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9d28c-5c28-4c6a-a2e3-17e897af4eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:54.544561 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78193b2c-a4b0-42af-bb4d-67f68808fcca None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:54.564410 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-78193b2c-a4b0-42af-bb4d-67f68808fcca None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 13:54:54.564410 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:54.564410 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:54:54.564410 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:54:54.564410 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:54:54.564410 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:54:54.564410 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:54:54.564410 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:54:54.564410 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 13:54:54.564410 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:54.564410 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:54:54.564410 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:54.564410 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:54.564410 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:54.564410 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:54.564410 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:54.564410 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:54.566464 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:54.566464 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:54.566464 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:54.566464 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:54.566464 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:54.566464 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:54.566464 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:54:54.566464 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:54:54.566464 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:54:54.566464 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:54:54.566464 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:54.566464 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:54.566464 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:54.566464 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:54.566464 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:54.566464 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:54.568456 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:54.568456 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:54.568456 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:54.568456 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:54.568456 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:54.568456 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:54.568456 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:54.568456 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:54.568456 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:54.568456 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:54.568456 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:54:54.568456 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:54:54.568456 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 13:54:54.568456 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:54.568456 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 442/884] 10.222.0.45 () {56 vars in 1126 bytes} [Mon Aug 1 13:54:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:54:54.572349 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78193b2c-a4b0-42af-bb4d-67f68808fcca None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:54.580364 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-cbb9d28c-5c28-4c6a-a2e3-17e897af4eac None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:54.582632 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-cbb9d28c-5c28-4c6a-a2e3-17e897af4eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:54.699855 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:54.732952 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 443/885] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:54 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:54:54.751576 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78193b2c-a4b0-42af-bb4d-67f68808fcca None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:54.777425 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9d28c-5c28-4c6a-a2e3-17e897af4eac None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:54.823377 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fc23da-a2e6-4c5f-bbba-8877ac87a7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:54.824425 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc23da-a2e6-4c5f-bbba-8877ac87a7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:54.825840 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc23da-a2e6-4c5f-bbba-8877ac87a7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:54.826303 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc23da-a2e6-4c5f-bbba-8877ac87a7e0 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:54.827544 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fc23da-a2e6-4c5f-bbba-8877ac87a7e0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:54.828150 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fc23da-a2e6-4c5f-bbba-8877ac87a7e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:54.830760 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fc23da-a2e6-4c5f-bbba-8877ac87a7e0 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:54.843000 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 443/886] 10.222.0.45 () {56 vars in 1122 bytes} [Mon Aug 1 13:54:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:54.854457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fc23da-a2e6-4c5f-bbba-8877ac87a7e0 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:54.956900 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:55.017195 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9d28c-5c28-4c6a-a2e3-17e897af4eac None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:55.072642 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fc23da-a2e6-4c5f-bbba-8877ac87a7e0 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:55.096097 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586198e9-85ce-4aaf-916b-a97fbb108628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:55.097243 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586198e9-85ce-4aaf-916b-a97fbb108628 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:55.097833 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586198e9-85ce-4aaf-916b-a97fbb108628 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:55.098690 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586198e9-85ce-4aaf-916b-a97fbb108628 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:55.099993 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586198e9-85ce-4aaf-916b-a97fbb108628 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:55.103807 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586198e9-85ce-4aaf-916b-a97fbb108628 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:55.145206 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 444/887] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Aug 1 13:54:54 2022] POST /identity/v3/services => generated 269 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 13:54:55.159917 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a651f-cb53-4c1b-aa5b-674ee8de79e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:55.161258 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a651f-cb53-4c1b-aa5b-674ee8de79e8 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:55.161768 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a651f-cb53-4c1b-aa5b-674ee8de79e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:55.162264 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a651f-cb53-4c1b-aa5b-674ee8de79e8 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:55.163649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93a651f-cb53-4c1b-aa5b-674ee8de79e8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:55.164265 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93a651f-cb53-4c1b-aa5b-674ee8de79e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:55.167168 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93a651f-cb53-4c1b-aa5b-674ee8de79e8 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:55.177855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 444/888] 10.222.0.45 () {56 vars in 1122 bytes} [Mon Aug 1 13:54:54 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:54:56.450969 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6267e22-7cb0-44e8-972c-a00cbb0f8d50 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:56.451805 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6267e22-7cb0-44e8-972c-a00cbb0f8d50 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:56.451945 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6267e22-7cb0-44e8-972c-a00cbb0f8d50 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:56.454838 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 445/889] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:56.463835 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e92ea91-482d-410e-92c5-8547d8d84162 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:56.464659 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e92ea91-482d-410e-92c5-8547d8d84162 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:56.464796 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e92ea91-482d-410e-92c5-8547d8d84162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:56.496486 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-9e92ea91-482d-410e-92c5-8547d8d84162 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:56.498505 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-9e92ea91-482d-410e-92c5-8547d8d84162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:56.633823 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 445/890] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:56 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:54:56.672485 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6267e22-7cb0-44e8-972c-a00cbb0f8d50 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:56.773667 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:56.876663 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6267e22-7cb0-44e8-972c-a00cbb0f8d50 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:56.946766 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45caf04b-e6c9-4212-9cd9-32cd725448fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:56.947878 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45caf04b-e6c9-4212-9cd9-32cd725448fe None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:56.948355 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45caf04b-e6c9-4212-9cd9-32cd725448fe None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:56.948816 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45caf04b-e6c9-4212-9cd9-32cd725448fe None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:56.970704 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45caf04b-e6c9-4212-9cd9-32cd725448fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:54:56.981474 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-45caf04b-e6c9-4212-9cd9-32cd725448fe None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 13:54:56.981474 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:56.981474 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:56.981474 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:56.981474 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:56.981474 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:56.981474 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:56.981474 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:56.981474 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:56.981474 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:56.981474 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:56.981474 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:56.981474 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:54:56.981474 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:54:56.981474 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:54:56.981474 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:54:56.983406 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:54:56.983406 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:54:56.983406 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:54:56.983406 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:54:56.983406 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:56.983406 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:56.983406 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:56.983406 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:56.983406 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:56.983406 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:56.983406 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:56.983406 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:56.983406 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:56.983406 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:56.983406 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:56.983406 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:56.985203 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:56.985203 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:56.985203 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:56.985203 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:56.985203 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:54:56.985203 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:54:56.985203 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:54:56.985203 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:54:56.985203 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:54:56.985203 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:54:56.985203 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 13:54:56.985203 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:56.985203 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 446/891] 10.222.0.45 () {56 vars in 1120 bytes} [Mon Aug 1 13:54:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:56.994168 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e92ea91-482d-410e-92c5-8547d8d84162 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:57.053649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:57.188710 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e92ea91-482d-410e-92c5-8547d8d84162 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:57.243451 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f51ce1-458f-4b06-a5bd-a5d1950b1024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:57.244559 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f51ce1-458f-4b06-a5bd-a5d1950b1024 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:57.245064 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f51ce1-458f-4b06-a5bd-a5d1950b1024 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:57.245546 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f51ce1-458f-4b06-a5bd-a5d1950b1024 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:57.246660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f51ce1-458f-4b06-a5bd-a5d1950b1024 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:57.246975 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f51ce1-458f-4b06-a5bd-a5d1950b1024 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:57.248466 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f51ce1-458f-4b06-a5bd-a5d1950b1024 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:57.258867 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 446/892] 10.222.0.45 () {56 vars in 1115 bytes} [Mon Aug 1 13:54:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:54:57.268202 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45caf04b-e6c9-4212-9cd9-32cd725448fe None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:57.441352 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45caf04b-e6c9-4212-9cd9-32cd725448fe None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:57.460494 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:57.507967 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66a2c07-325a-42a4-91d4-740a62be2b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:57.509073 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a2c07-325a-42a4-91d4-740a62be2b66 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:57.509554 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a2c07-325a-42a4-91d4-740a62be2b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:57.510031 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a2c07-325a-42a4-91d4-740a62be2b66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:57.531402 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66a2c07-325a-42a4-91d4-740a62be2b66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:57.534251 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66a2c07-325a-42a4-91d4-740a62be2b66 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:57.570716 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-a66a2c07-325a-42a4-91d4-740a62be2b66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:57.570716 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:57.570716 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:57.570716 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:57.570716 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:57.570716 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:57.570716 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:57.570716 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:57.570716 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:57.570716 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:57.570716 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:57.570716 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:57.570716 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:57.570716 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:57.570716 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:57.572258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:57.572258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:57.572258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:57.572258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:57.572258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:57.572258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:57.572258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:57.572258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:57.572258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:57.572258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:57.572258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:57.572258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:57.572258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:57.572258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:57.572258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:57.572258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:57.572258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:57.573380 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:57.573380 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:57.573380 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:57.573380 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:57.575882 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 447/893] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:57.583128 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f51ce1-458f-4b06-a5bd-a5d1950b1024 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:57.692069 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:57.703493 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f51ce1-458f-4b06-a5bd-a5d1950b1024 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:57.709973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc6413c-2e1f-48fe-ab23-c15115f16e61 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:57.710462 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc6413c-2e1f-48fe-ab23-c15115f16e61 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:57.710953 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc6413c-2e1f-48fe-ab23-c15115f16e61 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:57.720341 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 448/894] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:54:57.723169 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc90e9-a80b-4090-b67a-d0b4e65e25af None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:57.724048 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc90e9-a80b-4090-b67a-d0b4e65e25af None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:57.724848 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc90e9-a80b-4090-b67a-d0b4e65e25af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:57.754406 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7241b2-02b3-46d1-8fde-12ee54a279c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:57.755312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7241b2-02b3-46d1-8fde-12ee54a279c2 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:57.755778 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7241b2-02b3-46d1-8fde-12ee54a279c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:57.756234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7241b2-02b3-46d1-8fde-12ee54a279c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:57.757010 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7241b2-02b3-46d1-8fde-12ee54a279c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:57.757319 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7241b2-02b3-46d1-8fde-12ee54a279c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:57.758799 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7241b2-02b3-46d1-8fde-12ee54a279c2 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:57.761095 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c0fc90e9-a80b-4090-b67a-d0b4e65e25af None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:57.763665 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c0fc90e9-a80b-4090-b67a-d0b4e65e25af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:57.781645 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 447/895] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:57.790573 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7241b2-02b3-46d1-8fde-12ee54a279c2 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:57.906204 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 449/896] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:57 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:54:57.939727 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc90e9-a80b-4090-b67a-d0b4e65e25af None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:57.976656 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7241b2-02b3-46d1-8fde-12ee54a279c2 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:58.061049 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e44b34-1283-4f46-9ff2-3520c126803c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:58.062923 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e44b34-1283-4f46-9ff2-3520c126803c None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:58.064035 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e44b34-1283-4f46-9ff2-3520c126803c None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:58.065725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e44b34-1283-4f46-9ff2-3520c126803c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:58.084980 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e44b34-1283-4f46-9ff2-3520c126803c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:58.088158 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e44b34-1283-4f46-9ff2-3520c126803c None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:58.117202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-85e44b34-1283-4f46-9ff2-3520c126803c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 13:54:58.117202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:58.117202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:58.117202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:58.117202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:58.117202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:58.117202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:58.117202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:58.117202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:58.117202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:58.117202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:58.117202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:58.117202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:54:58.117202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:54:58.117202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:54:58.117202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:54:58.119181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:58.119181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:58.119181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:54:58.119181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:58.119181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:54:58.119181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:54:58.119181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:58.119181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:58.119181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:58.119181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:58.119181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:58.119181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:58.119181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:58.119181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:58.119181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:58.119181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:58.119181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:58.119181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:58.121194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:58.121194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:58.121194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:54:58.121194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:54:58.121194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:54:58.121194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:54:58.121194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:54:58.121194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:54:58.121194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 13:54:58.121194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:54:58.127511 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 448/897] 10.222.0.45 () {56 vars in 1117 bytes} [Mon Aug 1 13:54:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:54:58.129457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e44b34-1283-4f46-9ff2-3520c126803c None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:58.151943 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:58.157366 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:58.176194 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc90e9-a80b-4090-b67a-d0b4e65e25af None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:58.246953 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074a193-a4dd-463f-9135-db2bfd302a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:58.248651 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074a193-a4dd-463f-9135-db2bfd302a62 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:58.249468 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074a193-a4dd-463f-9135-db2bfd302a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:58.250297 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074a193-a4dd-463f-9135-db2bfd302a62 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:58.251757 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5074a193-a4dd-463f-9135-db2bfd302a62 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:58.254899 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5074a193-a4dd-463f-9135-db2bfd302a62 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:58.274305 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-5074a193-a4dd-463f-9135-db2bfd302a62 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 13:54:58.274305 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:58.274305 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:54:58.274305 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:54:58.274305 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:54:58.274305 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:54:58.274305 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:54:58.274305 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:54:58.274305 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 13:54:58.274305 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:58.274305 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:54:58.274305 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:58.274305 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:58.274305 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:58.274305 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:58.274305 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:58.274305 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:58.276494 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:58.276494 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:58.276494 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:58.276494 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:58.276494 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:58.276494 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:58.276494 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:54:58.276494 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:54:58.276494 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:54:58.276494 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:54:58.276494 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:58.276494 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:58.276494 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:58.276494 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:58.276494 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:58.276494 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:58.278388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:58.278388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:58.278388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:58.278388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:58.278388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:58.278388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:58.278388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:58.278388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:58.278388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:58.278388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:58.278388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:54:58.278388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:54:58.278388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 13:54:58.278388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:58.283144 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 450/898] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 13:54:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:54:58.293854 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e44b34-1283-4f46-9ff2-3520c126803c None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:58.294129 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074a193-a4dd-463f-9135-db2bfd302a62 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:58.333548 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8796049-60c6-43f6-aaf6-1ee8716dfe95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:58.334195 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8796049-60c6-43f6-aaf6-1ee8716dfe95 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:58.334463 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8796049-60c6-43f6-aaf6-1ee8716dfe95 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:58.334774 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8796049-60c6-43f6-aaf6-1ee8716dfe95 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:58.335587 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8796049-60c6-43f6-aaf6-1ee8716dfe95 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:58.335885 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8796049-60c6-43f6-aaf6-1ee8716dfe95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:58.337404 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8796049-60c6-43f6-aaf6-1ee8716dfe95 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:58.366405 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 449/899] 10.222.0.45 () {56 vars in 1149 bytes} [Mon Aug 1 13:54:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:58.377150 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8796049-60c6-43f6-aaf6-1ee8716dfe95 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:58.514222 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074a193-a4dd-463f-9135-db2bfd302a62 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:58.523948 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:58.601555 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110fbe26-e494-4c56-b71b-d6871d8dc6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:58.602372 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110fbe26-e494-4c56-b71b-d6871d8dc6e9 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:58.602891 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110fbe26-e494-4c56-b71b-d6871d8dc6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:58.603321 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110fbe26-e494-4c56-b71b-d6871d8dc6e9 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:58.604151 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110fbe26-e494-4c56-b71b-d6871d8dc6e9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:58.604623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110fbe26-e494-4c56-b71b-d6871d8dc6e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:58.606212 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110fbe26-e494-4c56-b71b-d6871d8dc6e9 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:58.606622 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8796049-60c6-43f6-aaf6-1ee8716dfe95 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:58.620648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 451/900] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 13:54:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:58.620648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36d313-377d-43db-8942-dfa68c0921a7 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:58.620648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36d313-377d-43db-8942-dfa68c0921a7 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:58.620648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36d313-377d-43db-8942-dfa68c0921a7 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:58.620648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 452/901] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:58.628623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45224eaf-9ed9-4dd0-8360-b0bba459841a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:58.628890 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45224eaf-9ed9-4dd0-8360-b0bba459841a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:58.629200 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45224eaf-9ed9-4dd0-8360-b0bba459841a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:58.650729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-45224eaf-9ed9-4dd0-8360-b0bba459841a None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:58.652174 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-45224eaf-9ed9-4dd0-8360-b0bba459841a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:58.671007 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:58.677058 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c9da6-4cfb-4dca-b589-236f982a78cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:58.678367 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c9da6-4cfb-4dca-b589-236f982a78cc None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:58.678605 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c9da6-4cfb-4dca-b589-236f982a78cc None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:58.679110 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c9da6-4cfb-4dca-b589-236f982a78cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:58.762279 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 453/902] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:58 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:54:58.763159 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45224eaf-9ed9-4dd0-8360-b0bba459841a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:58.764109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c9da6-4cfb-4dca-b589-236f982a78cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:58.765785 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c9da6-4cfb-4dca-b589-236f982a78cc None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:58.857269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:58.917401 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 450/903] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:54:58 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:54:58.919577 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45224eaf-9ed9-4dd0-8360-b0bba459841a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:58.919776 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6755244-2a6c-479a-9ff7-cfdc483e5150 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:58.920216 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6755244-2a6c-479a-9ff7-cfdc483e5150 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:58.920749 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6755244-2a6c-479a-9ff7-cfdc483e5150 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:58.923067 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 451/904] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:54:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:58.924312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6755244-2a6c-479a-9ff7-cfdc483e5150 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:58.985551 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d36918-282a-4165-9365-71edc13f3ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:58.986614 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d36918-282a-4165-9365-71edc13f3ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:58.987117 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d36918-282a-4165-9365-71edc13f3ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:58.987617 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d36918-282a-4165-9365-71edc13f3ad5 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:58.988823 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d36918-282a-4165-9365-71edc13f3ad5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:58.989349 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d36918-282a-4165-9365-71edc13f3ad5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:58.991887 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d36918-282a-4165-9365-71edc13f3ad5 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:59.009781 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 454/905] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 13:54:58 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:59.009781 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d36918-282a-4165-9365-71edc13f3ad5 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:59.041284 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:59.130719 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6755244-2a6c-479a-9ff7-cfdc483e5150 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:59.183700 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e103b58-6893-4af6-852d-78753180ced4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:59.184577 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e103b58-6893-4af6-852d-78753180ced4 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:59.185019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e103b58-6893-4af6-852d-78753180ced4 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:59.185440 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e103b58-6893-4af6-852d-78753180ced4 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:59.186375 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e103b58-6893-4af6-852d-78753180ced4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:59.188054 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e103b58-6893-4af6-852d-78753180ced4 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:59.210171 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 452/906] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Aug 1 13:54:58 2022] POST /identity/v3/services => generated 241 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:54:59.212635 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d42a2-6aad-4f99-910c-23668337a31e None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:59.213090 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d42a2-6aad-4f99-910c-23668337a31e None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:59.213622 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d42a2-6aad-4f99-910c-23668337a31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:59.220576 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d36918-282a-4165-9365-71edc13f3ad5 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:59.251994 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-457d42a2-6aad-4f99-910c-23668337a31e None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:59.254262 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-457d42a2-6aad-4f99-910c-23668337a31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:59.302345 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:59.304752 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d49dd5-6806-4ec8-86bf-2b51fc996c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:59.306285 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d49dd5-6806-4ec8-86bf-2b51fc996c69 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:59.306727 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d49dd5-6806-4ec8-86bf-2b51fc996c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:59.307210 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d49dd5-6806-4ec8-86bf-2b51fc996c69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:59.332668 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d49dd5-6806-4ec8-86bf-2b51fc996c69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:59.335481 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d49dd5-6806-4ec8-86bf-2b51fc996c69 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:59.364227 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-29d49dd5-6806-4ec8-86bf-2b51fc996c69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:59.364227 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:59.364227 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:59.364227 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:59.364227 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:59.364227 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:59.364227 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:59.364227 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:59.364227 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:59.364227 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:59.364227 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:59.364227 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:59.364227 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:59.364227 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:59.364227 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:59.365965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:59.365965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:59.365965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:59.365965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:59.365965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:59.365965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:59.365965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:59.365965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:59.365965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:59.365965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:59.365965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:59.365965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:59.365965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:59.365965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:59.365965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:59.365965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:59.365965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:59.367623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:59.367623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:59.367623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:59.367623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:54:59.367623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 455/907] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:59.367623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d49dd5-6806-4ec8-86bf-2b51fc996c69 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:59.412315 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 453/908] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:54:59.412315 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d42a2-6aad-4f99-910c-23668337a31e None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:59.493353 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:59.584300 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d49dd5-6806-4ec8-86bf-2b51fc996c69 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:59.609252 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d42a2-6aad-4f99-910c-23668337a31e None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:59.646767 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484a331-1538-4479-99c0-8de1f1b6fc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:59.647814 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484a331-1538-4479-99c0-8de1f1b6fc6b None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:59.648348 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484a331-1538-4479-99c0-8de1f1b6fc6b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:59.648831 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484a331-1538-4479-99c0-8de1f1b6fc6b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:59.650078 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484a331-1538-4479-99c0-8de1f1b6fc6b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:59.652688 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484a331-1538-4479-99c0-8de1f1b6fc6b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:59.679465 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73500819-c3e9-407c-9241-af3470bc4cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:59.680809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73500819-c3e9-407c-9241-af3470bc4cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:59.681309 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73500819-c3e9-407c-9241-af3470bc4cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:59.681812 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73500819-c3e9-407c-9241-af3470bc4cf6 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:59.683148 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73500819-c3e9-407c-9241-af3470bc4cf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:59.683689 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73500819-c3e9-407c-9241-af3470bc4cf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:59.686423 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73500819-c3e9-407c-9241-af3470bc4cf6 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:59.700693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 456/909] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 13:54:59 2022] POST /identity/v3/endpoints => generated 346 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:54:59.700693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484a331-1538-4479-99c0-8de1f1b6fc6b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:59.724609 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 454/910] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:59.735118 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73500819-c3e9-407c-9241-af3470bc4cf6 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:59.791619 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:59.936795 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484a331-1538-4479-99c0-8de1f1b6fc6b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:59.968490 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:59.986285 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73500819-c3e9-407c-9241-af3470bc4cf6 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:00.024481 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a73d7-0df8-4590-9c9c-96ecd2c0f277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:00.025959 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a73d7-0df8-4590-9c9c-96ecd2c0f277 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:00.026673 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a73d7-0df8-4590-9c9c-96ecd2c0f277 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:00.027379 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a73d7-0df8-4590-9c9c-96ecd2c0f277 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:00.028858 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336a73d7-0df8-4590-9c9c-96ecd2c0f277 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:00.031936 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336a73d7-0df8-4590-9c9c-96ecd2c0f277 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:00.051510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-336a73d7-0df8-4590-9c9c-96ecd2c0f277 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 13:55:00.051510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:00.051510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:55:00.051510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:55:00.051510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:55:00.051510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:55:00.051510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:55:00.051510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:55:00.051510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 13:55:00.051510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:00.051510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:55:00.051510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:00.051510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:00.051510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:00.051510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:00.051510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:00.053865 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:00.053865 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:00.053865 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:00.053865 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:00.053865 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:00.053865 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:00.053865 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:00.053865 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:55:00.053865 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:55:00.053865 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:55:00.053865 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:55:00.053865 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:00.053865 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:00.053865 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:00.053865 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:00.053865 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:00.055758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:00.055758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:00.055758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:00.055758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:00.055758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:00.055758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:00.055758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:00.055758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:00.055758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:00.055758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:00.055758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:00.055758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:55:00.055758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:55:00.055758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 13:55:00.055758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:00.059050 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 457/911] 10.222.0.45 () {56 vars in 1132 bytes} [Mon Aug 1 13:54:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:00.066094 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a44e77-63f3-4d36-829b-fd8fbebb384d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:00.068050 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a44e77-63f3-4d36-829b-fd8fbebb384d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:00.068050 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a44e77-63f3-4d36-829b-fd8fbebb384d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:00.068050 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a44e77-63f3-4d36-829b-fd8fbebb384d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:00.068919 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a73d7-0df8-4590-9c9c-96ecd2c0f277 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:00.083904 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a44e77-63f3-4d36-829b-fd8fbebb384d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:00.086711 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a44e77-63f3-4d36-829b-fd8fbebb384d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:00.112473 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-b8a44e77-63f3-4d36-829b-fd8fbebb384d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:55:00.112473 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:00.112473 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:00.112473 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:00.112473 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:00.112473 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:00.112473 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:00.112473 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:00.112473 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:00.112473 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:00.112473 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:00.112473 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:00.112473 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:55:00.112473 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:55:00.112473 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:55:00.114284 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:55:00.114284 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:00.114284 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:00.114284 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:00.114284 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:00.114284 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:00.114284 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:00.114284 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:00.114284 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:00.114284 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:00.114284 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:00.114284 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:00.114284 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:00.114284 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:00.114284 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:00.114284 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:00.114284 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:00.116155 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:55:00.116155 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:55:00.116155 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:55:00.116155 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:55:00.116155 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:55:00.116155 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:55:00.116155 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:55:00.116155 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:00.120492 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 455/912] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:54:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:00.123439 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a44e77-63f3-4d36-829b-fd8fbebb384d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:00.241713 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:00.284646 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a44e77-63f3-4d36-829b-fd8fbebb384d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:00.294919 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a73d7-0df8-4590-9c9c-96ecd2c0f277 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:00.355182 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f796cff-ef6e-454d-9a8e-2ee4255aadf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:00.356490 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f796cff-ef6e-454d-9a8e-2ee4255aadf0 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:00.357078 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f796cff-ef6e-454d-9a8e-2ee4255aadf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:00.357624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f796cff-ef6e-454d-9a8e-2ee4255aadf0 None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:00.358639 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f796cff-ef6e-454d-9a8e-2ee4255aadf0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:00.359280 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f796cff-ef6e-454d-9a8e-2ee4255aadf0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:00.361215 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f796cff-ef6e-454d-9a8e-2ee4255aadf0 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:00.364945 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f535f3-096b-4440-b282-886f7b7ead84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:00.366179 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f535f3-096b-4440-b282-886f7b7ead84 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:00.366797 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f535f3-096b-4440-b282-886f7b7ead84 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:00.367407 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f535f3-096b-4440-b282-886f7b7ead84 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:00.369066 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f535f3-096b-4440-b282-886f7b7ead84 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:00.369806 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f535f3-096b-4440-b282-886f7b7ead84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:00.372660 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f535f3-096b-4440-b282-886f7b7ead84 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:00.383058 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 456/913] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:55:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:00.384955 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f796cff-ef6e-454d-9a8e-2ee4255aadf0 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:00.388511 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 458/914] 10.222.0.45 () {56 vars in 1126 bytes} [Mon Aug 1 13:55:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:00.393979 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f535f3-096b-4440-b282-886f7b7ead84 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:00.473266 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:00.553632 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f796cff-ef6e-454d-9a8e-2ee4255aadf0 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:00.578066 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f535f3-096b-4440-b282-886f7b7ead84 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:00.609408 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77daa30-007e-4654-98f4-dd93b32ad09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:00.611061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77daa30-007e-4654-98f4-dd93b32ad09f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:00.611552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77daa30-007e-4654-98f4-dd93b32ad09f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:00.612052 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77daa30-007e-4654-98f4-dd93b32ad09f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:00.614192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77daa30-007e-4654-98f4-dd93b32ad09f None admin] RBAC: Adding query filter params (role.id=c29b2bae1d6c4e438cb861ed195d5c72, scope.project.id=36f063d8f65d494a89dbbd72c53417c8, user.id=e07e83c0bdba47d39c09308eba1bdaae) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:00.614748 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77daa30-007e-4654-98f4-dd93b32ad09f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:00.619221 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77daa30-007e-4654-98f4-dd93b32ad09f None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:00.624686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db10ad5-f9d3-4ca6-9e10-f41248c147b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:00.626270 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db10ad5-f9d3-4ca6-9e10-f41248c147b3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:00.626740 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db10ad5-f9d3-4ca6-9e10-f41248c147b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:00.627224 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db10ad5-f9d3-4ca6-9e10-f41248c147b3 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:00.628094 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db10ad5-f9d3-4ca6-9e10-f41248c147b3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:00.628625 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db10ad5-f9d3-4ca6-9e10-f41248c147b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:00.630214 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db10ad5-f9d3-4ca6-9e10-f41248c147b3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:00.632372 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 457/915] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:55:00 2022] GET /identity/v3/role_assignments?user.id=e07e83c0bdba47d39c09308eba1bdaae&scope.project.id=36f063d8f65d494a89dbbd72c53417c8&role.id=c29b2bae1d6c4e438cb861ed195d5c72 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:00.642124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 459/916] 10.222.0.45 () {56 vars in 1126 bytes} [Mon Aug 1 13:55:00 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:00.649489 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77daa30-007e-4654-98f4-dd93b32ad09f None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:00.653220 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:00.849885 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77daa30-007e-4654-98f4-dd93b32ad09f None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:00.918128 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4c823-6aed-4c48-8a05-619e7b792ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:00.919181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4c823-6aed-4c48-8a05-619e7b792ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:00.919654 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4c823-6aed-4c48-8a05-619e7b792ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:00.920125 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4c823-6aed-4c48-8a05-619e7b792ea2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:00.921543 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4c823-6aed-4c48-8a05-619e7b792ea2 None admin] RBAC: Adding query filter params (interface=public, service_id=eaf201dbcabd485b8ecdae07103c23a1, region_id=RegionOne) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:00.922258 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4c823-6aed-4c48-8a05-619e7b792ea2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:00.924818 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4c823-6aed-4c48-8a05-619e7b792ea2 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:00.935332 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 458/917] 10.222.0.45 () {56 vars in 1251 bytes} [Mon Aug 1 13:55:00 2022] GET /identity/v3/endpoints?service_id=eaf201dbcabd485b8ecdae07103c23a1&interface=public®ion_id=RegionOne => generated 194 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:00.944722 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db10ad5-f9d3-4ca6-9e10-f41248c147b3 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:01.152307 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db10ad5-f9d3-4ca6-9e10-f41248c147b3 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:01.156983 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:01.218390 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ac2db-1c39-4d98-a1d1-06cd3ae3cff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:01.219666 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ac2db-1c39-4d98-a1d1-06cd3ae3cff3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:01.220173 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ac2db-1c39-4d98-a1d1-06cd3ae3cff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:01.220684 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ac2db-1c39-4d98-a1d1-06cd3ae3cff3 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:01.221848 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ac2db-1c39-4d98-a1d1-06cd3ae3cff3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:01.224423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ac2db-1c39-4d98-a1d1-06cd3ae3cff3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:01.237194 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 460/918] 10.222.0.45 () {56 vars in 1087 bytes} [Mon Aug 1 13:55:00 2022] GET /identity/v3/services => generated 1497 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 13:55:02.541030 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927eb77a-00ac-4e74-bde8-e79322925dee None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:02.541928 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927eb77a-00ac-4e74-bde8-e79322925dee None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:02.541928 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927eb77a-00ac-4e74-bde8-e79322925dee None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:02.546893 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 459/919] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:02.549967 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b4710-4eff-4b3b-8070-67251c1e2b5f None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:02.550792 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b4710-4eff-4b3b-8070-67251c1e2b5f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:02.550792 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b4710-4eff-4b3b-8070-67251c1e2b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:02.585000 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-230b4710-4eff-4b3b-8070-67251c1e2b5f None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:02.587192 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-230b4710-4eff-4b3b-8070-67251c1e2b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:02.729171 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 461/920] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:55:02.768351 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927eb77a-00ac-4e74-bde8-e79322925dee None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:02.924993 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:02.990459 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaec1c1-e167-40a8-bfd9-07343be331a1 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:02.990941 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaec1c1-e167-40a8-bfd9-07343be331a1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:02.991186 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaec1c1-e167-40a8-bfd9-07343be331a1 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:02.993538 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 462/921] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:55:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:03.002149 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927eb77a-00ac-4e74-bde8-e79322925dee None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:03.002414 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ea56b-fd44-46fe-b1a7-e9cf7c86e86a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:03.002911 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ea56b-fd44-46fe-b1a7-e9cf7c86e86a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:03.003391 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ea56b-fd44-46fe-b1a7-e9cf7c86e86a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:03.028851 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-621ea56b-fd44-46fe-b1a7-e9cf7c86e86a None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:03.030065 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-621ea56b-fd44-46fe-b1a7-e9cf7c86e86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:03.073089 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29102d44-9cbf-4619-970c-cfe7058dc656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:03.074670 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29102d44-9cbf-4619-970c-cfe7058dc656 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:03.075171 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29102d44-9cbf-4619-970c-cfe7058dc656 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:03.075693 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29102d44-9cbf-4619-970c-cfe7058dc656 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:03.077062 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29102d44-9cbf-4619-970c-cfe7058dc656 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:03.079970 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29102d44-9cbf-4619-970c-cfe7058dc656 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:03.102587 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-29102d44-9cbf-4619-970c-cfe7058dc656 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 13:55:03.102587 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:03.102587 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:55:03.102587 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:55:03.102587 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:55:03.102587 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:55:03.102587 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:55:03.102587 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:55:03.102587 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 13:55:03.102587 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:03.102587 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:55:03.102587 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:03.102587 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:03.102587 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:03.102587 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:03.102587 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:03.102587 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:03.104509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:03.104509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:03.104509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:03.104509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:03.104509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:03.104509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:03.104509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:55:03.104509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:55:03.104509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:55:03.104509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:55:03.104509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:03.104509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:03.104509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:03.104509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:03.104509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:03.104509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:03.106083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:03.106083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:03.106083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:03.106083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:03.106083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:03.106083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:03.106083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:03.106083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:03.106083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:03.106083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:03.106083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:55:03.106083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:55:03.106083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 13:55:03.106083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:03.106083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 460/922] 10.222.0.45 () {56 vars in 1126 bytes} [Mon Aug 1 13:55:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:03.120315 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29102d44-9cbf-4619-970c-cfe7058dc656 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:03.180923 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 463/923] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:55:03.222244 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ea56b-fd44-46fe-b1a7-e9cf7c86e86a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:03.279486 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:03.346279 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29102d44-9cbf-4619-970c-cfe7058dc656 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:03.416713 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:03.422051 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f089d9a-6a1e-4cf7-8740-9110f25f03dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:03.424132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f089d9a-6a1e-4cf7-8740-9110f25f03dd None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:03.424991 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f089d9a-6a1e-4cf7-8740-9110f25f03dd None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:03.425685 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f089d9a-6a1e-4cf7-8740-9110f25f03dd None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:03.427246 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f089d9a-6a1e-4cf7-8740-9110f25f03dd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:03.428126 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f089d9a-6a1e-4cf7-8740-9110f25f03dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:03.431179 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f089d9a-6a1e-4cf7-8740-9110f25f03dd None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:03.433575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ea56b-fd44-46fe-b1a7-e9cf7c86e86a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:03.442446 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 461/924] 10.222.0.45 () {56 vars in 1122 bytes} [Mon Aug 1 13:55:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:03.452845 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f089d9a-6a1e-4cf7-8740-9110f25f03dd None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:03.508164 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ebedd4-9c57-442f-9b1c-ae12fd0ff040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:03.509330 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ebedd4-9c57-442f-9b1c-ae12fd0ff040 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:03.509896 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ebedd4-9c57-442f-9b1c-ae12fd0ff040 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:03.510470 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ebedd4-9c57-442f-9b1c-ae12fd0ff040 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:03.511825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ebedd4-9c57-442f-9b1c-ae12fd0ff040 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:03.514764 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ebedd4-9c57-442f-9b1c-ae12fd0ff040 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:03.519363 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: WARNING keystone.server.flask.common [None req-e9ebedd4-9c57-442f-9b1c-ae12fd0ff040 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 01 13:55:03.554964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.sql.core [None req-e9ebedd4-9c57-442f-9b1c-ae12fd0ff040 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 13:55:03.554964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 01 13:55:03.554964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [parameters: {'id': 'edf421b0ddcb4534b462db5cf20350a0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 01 13:55:03.554964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80311) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 13:55:03.579395 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-e9ebedd4-9c57-442f-9b1c-ae12fd0ff040 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id edf421b0ddcb4534b462db5cf20350a0.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id edf421b0ddcb4534b462db5cf20350a0. Aug 01 13:55:03.579395 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:03.579395 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 13:55:03.579395 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 13:55:03.579395 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 13:55:03.579395 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 13:55:03.579395 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 13:55:03.579395 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 13:55:03.579395 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 13:55:03.579395 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application conn.query(q) Aug 01 13:55:03.579395 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 13:55:03.579395 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 13:55:03.579395 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 13:55:03.581148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application result.read() Aug 01 13:55:03.581148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 13:55:03.581148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 13:55:03.581148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 13:55:03.581148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 13:55:03.581148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 13:55:03.581148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 13:55:03.581148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 13:55:03.581148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 13:55:03.581148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 13:55:03.581148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:03.581148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 13:55:03.581148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:03.581148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:03.581148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 13:55:03.581148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 13:55:03.581148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 01 13:55:03.581148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 01 13:55:03.583056 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 13:55:03.583056 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application next(self.gen) Aug 01 13:55:03.583056 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 13:55:03.583056 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application yield resource Aug 01 13:55:03.583056 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 13:55:03.583056 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application next(self.gen) Aug 01 13:55:03.583056 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 13:55:03.583056 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 13:55:03.583056 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 13:55:03.583056 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 13:55:03.583056 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 13:55:03.583056 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise self.value Aug 01 13:55:03.583056 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 13:55:03.583056 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 13:55:03.583056 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 13:55:03.583056 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application session.commit() Aug 01 13:55:03.583056 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 13:55:03.587434 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 13:55:03.587434 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 13:55:03.587434 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 13:55:03.587434 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 13:55:03.587434 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self.session.flush() Aug 01 13:55:03.587434 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 13:55:03.587434 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 13:55:03.587434 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 13:55:03.587434 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 13:55:03.587434 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 13:55:03.587434 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application compat.raise_( Aug 01 13:55:03.587434 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 13:55:03.587434 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception Aug 01 13:55:03.587434 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 13:55:03.587434 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 13:55:03.587434 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 13:55:03.587434 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 13:55:03.591071 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 13:55:03.591071 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 13:55:03.591071 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 13:55:03.591071 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 13:55:03.591071 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 01 13:55:03.591071 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 01 13:55:03.591071 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 13:55:03.591071 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 13:55:03.591071 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 13:55:03.591071 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 13:55:03.591071 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 13:55:03.591071 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 13:55:03.591071 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 13:55:03.591071 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 13:55:03.591071 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 13:55:03.595201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 13:55:03.595201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 13:55:03.595201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception Aug 01 13:55:03.595201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 13:55:03.595201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 13:55:03.595201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 13:55:03.595201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 13:55:03.595201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 13:55:03.595201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 13:55:03.595201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 13:55:03.595201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application conn.query(q) Aug 01 13:55:03.595201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 13:55:03.595201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 13:55:03.595201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 13:55:03.595201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application result.read() Aug 01 13:55:03.595201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 13:55:03.595201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 13:55:03.598923 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 13:55:03.598923 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 13:55:03.598923 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 13:55:03.598923 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 13:55:03.598923 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 13:55:03.598923 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 13:55:03.598923 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 13:55:03.598923 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 01 13:55:03.598923 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [parameters: {'id': 'edf421b0ddcb4534b462db5cf20350a0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 01 13:55:03.598923 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 13:55:03.598923 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:03.598923 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:55:03.598923 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:03.598923 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:03.598923 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 01 13:55:03.602513 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 01 13:55:03.602513 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 13:55:03.602513 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 13:55:03.602513 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 01 13:55:03.602513 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:03.602513 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:55:03.602513 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:03.602513 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:03.602513 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:03.602513 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:03.602513 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:03.602513 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:03.602513 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:03.602513 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:03.602513 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:03.602513 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:03.602513 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:03.606302 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:03.606302 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 01 13:55:03.606302 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 01 13:55:03.606302 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:03.606302 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:03.606302 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 01 13:55:03.606302 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 01 13:55:03.606302 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id edf421b0ddcb4534b462db5cf20350a0. Aug 01 13:55:03.606302 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:03.606302 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 464/925] 10.222.0.45 () {60 vars in 1140 bytes} [Mon Aug 1 13:55:03 2022] POST /identity/v3/projects => generated 262 bytes in 360 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:55:03.606302 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ebedd4-9c57-442f-9b1c-ae12fd0ff040 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:03.699179 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f089d9a-6a1e-4cf7-8740-9110f25f03dd None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:03.769195 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:03.775606 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2f87e-93ac-429c-9e01-7fed739bc9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:03.777934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2f87e-93ac-429c-9e01-7fed739bc9fc None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:03.778870 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2f87e-93ac-429c-9e01-7fed739bc9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:03.779629 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2f87e-93ac-429c-9e01-7fed739bc9fc None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:03.781391 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d2f87e-93ac-429c-9e01-7fed739bc9fc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:03.782324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d2f87e-93ac-429c-9e01-7fed739bc9fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:03.785442 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d2f87e-93ac-429c-9e01-7fed739bc9fc None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:03.797082 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 462/926] 10.222.0.45 () {56 vars in 1122 bytes} [Mon Aug 1 13:55:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:03.809940 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2f87e-93ac-429c-9e01-7fed739bc9fc None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:03.812000 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ebedd4-9c57-442f-9b1c-ae12fd0ff040 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:03.874547 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4759cdee-2a70-4a69-871e-9b671e077018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:03.875575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4759cdee-2a70-4a69-871e-9b671e077018 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:03.875998 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4759cdee-2a70-4a69-871e-9b671e077018 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:03.877020 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4759cdee-2a70-4a69-871e-9b671e077018 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:03.889162 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4759cdee-2a70-4a69-871e-9b671e077018 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:03.892975 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4759cdee-2a70-4a69-871e-9b671e077018 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:03.917326 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-4759cdee-2a70-4a69-871e-9b671e077018 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:55:03.917326 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:03.917326 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:03.917326 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:03.917326 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:03.917326 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:03.917326 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:03.917326 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:03.917326 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:03.917326 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:03.917326 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:03.917326 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:03.917326 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:55:03.917326 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:55:03.917326 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:55:03.919213 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:55:03.919213 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:03.919213 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:03.919213 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:03.919213 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:03.919213 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:03.919213 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:03.919213 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:03.919213 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:03.919213 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:03.919213 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:03.919213 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:03.919213 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:03.919213 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:03.919213 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:03.919213 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:03.919213 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:03.921073 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:55:03.921073 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:55:03.921073 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:55:03.921073 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:55:03.921073 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:55:03.921073 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:55:03.921073 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:55:03.921073 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:03.921073 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 465/927] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:55:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:03.928600 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4759cdee-2a70-4a69-871e-9b671e077018 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:03.943301 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:04.033488 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2f87e-93ac-429c-9e01-7fed739bc9fc None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:04.116762 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81729c-6968-44fd-9237-7f9d5a98bd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:04.117807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81729c-6968-44fd-9237-7f9d5a98bd29 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:04.118265 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81729c-6968-44fd-9237-7f9d5a98bd29 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:04.118714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81729c-6968-44fd-9237-7f9d5a98bd29 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:04.120100 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81729c-6968-44fd-9237-7f9d5a98bd29 None admin] RBAC: Adding query filter params (interface=public, service_id=0e1e13fec9324c1bacd27cdae5829d49, region_id=RegionOne) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:04.120784 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81729c-6968-44fd-9237-7f9d5a98bd29 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:04.123420 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81729c-6968-44fd-9237-7f9d5a98bd29 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:04.134928 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 463/928] 10.222.0.45 () {56 vars in 1251 bytes} [Mon Aug 1 13:55:03 2022] GET /identity/v3/endpoints?service_id=0e1e13fec9324c1bacd27cdae5829d49&interface=public®ion_id=RegionOne => generated 194 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:04.142802 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf43a2-822b-4da7-9867-7fcde6eb3588 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:04.142802 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf43a2-822b-4da7-9867-7fcde6eb3588 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:04.142802 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf43a2-822b-4da7-9867-7fcde6eb3588 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:04.144777 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 464/929] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:55:04 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:04.148358 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622865ab-6c90-499c-94a6-61fcdaba4036 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:04.149179 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622865ab-6c90-499c-94a6-61fcdaba4036 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:04.149650 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622865ab-6c90-499c-94a6-61fcdaba4036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:04.157824 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4759cdee-2a70-4a69-871e-9b671e077018 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:04.171715 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-622865ab-6c90-499c-94a6-61fcdaba4036 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:04.173106 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-622865ab-6c90-499c-94a6-61fcdaba4036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:04.230135 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08545f-8e70-4153-931e-6c10df161a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:04.231210 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08545f-8e70-4153-931e-6c10df161a9a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:04.231690 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08545f-8e70-4153-931e-6c10df161a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:04.232176 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08545f-8e70-4153-931e-6c10df161a9a None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:04.233587 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08545f-8e70-4153-931e-6c10df161a9a None admin] RBAC: Adding query filter params (name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:04.234204 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08545f-8e70-4153-931e-6c10df161a9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:04.243914 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08545f-8e70-4153-931e-6c10df161a9a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:04.292452 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 466/930] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Aug 1 13:55:03 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:04.292452 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08545f-8e70-4153-931e-6c10df161a9a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:04.295786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 465/931] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:55:04.299828 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:04.320885 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622865ab-6c90-499c-94a6-61fcdaba4036 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:04.401189 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:04.474309 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08545f-8e70-4153-931e-6c10df161a9a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:04.497556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622865ab-6c90-499c-94a6-61fcdaba4036 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:04.535281 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae1d47-2035-4915-8eff-d8de7fbb3451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:04.536742 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae1d47-2035-4915-8eff-d8de7fbb3451 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:04.537415 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae1d47-2035-4915-8eff-d8de7fbb3451 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:04.538005 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae1d47-2035-4915-8eff-d8de7fbb3451 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:04.539361 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae1d47-2035-4915-8eff-d8de7fbb3451 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:04.540998 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae1d47-2035-4915-8eff-d8de7fbb3451 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:04.554032 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 467/932] 10.222.0.45 () {56 vars in 1087 bytes} [Mon Aug 1 13:55:04 2022] GET /identity/v3/services => generated 1497 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 13:55:04.582860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb750089-cbf7-4e8c-a97b-aeccfa77893c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:04.583563 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb750089-cbf7-4e8c-a97b-aeccfa77893c None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:04.583885 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb750089-cbf7-4e8c-a97b-aeccfa77893c None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:04.584373 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb750089-cbf7-4e8c-a97b-aeccfa77893c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:04.603405 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb750089-cbf7-4e8c-a97b-aeccfa77893c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:04.613526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-eb750089-cbf7-4e8c-a97b-aeccfa77893c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 13:55:04.613526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:04.613526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:04.613526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:04.613526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:04.613526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:04.613526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:04.613526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:04.613526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:04.613526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:04.613526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:04.613526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:04.613526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:04.613526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:04.613526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:04.613526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:04.615293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:04.615293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:04.615293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:04.615293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:04.615293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:04.615293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:04.615293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:04.615293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:04.615293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:04.615293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:04.615293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:04.615293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:04.615293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:04.615293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:04.615293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:04.615293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:04.616772 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:04.616772 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:04.616772 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:04.616772 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:04.616772 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:04.616772 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:04.616772 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:04.616772 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:04.616772 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:04.616772 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:04.616772 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 13:55:04.616772 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:04.618007 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 466/933] 10.222.0.45 () {56 vars in 1120 bytes} [Mon Aug 1 13:55:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:55:04.623913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae1d47-2035-4915-8eff-d8de7fbb3451 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:04.760826 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:04.809228 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30853417-c51f-40d4-8f2e-036d71f8df81 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:04.809947 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30853417-c51f-40d4-8f2e-036d71f8df81 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:04.809947 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30853417-c51f-40d4-8f2e-036d71f8df81 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:04.816686 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 467/934] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:55:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:04.822018 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eeab8c-7696-422d-b47d-062ea66cae2f None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:04.823055 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eeab8c-7696-422d-b47d-062ea66cae2f None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:04.823055 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eeab8c-7696-422d-b47d-062ea66cae2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:04.847305 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae1d47-2035-4915-8eff-d8de7fbb3451 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:04.856696 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-c4eeab8c-7696-422d-b47d-062ea66cae2f None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:04.858930 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c4eeab8c-7696-422d-b47d-062ea66cae2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:04.939407 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f7cc1-fd60-409a-a93f-58f6c3237ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:04.941258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f7cc1-fd60-409a-a93f-58f6c3237ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:04.942111 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f7cc1-fd60-409a-a93f-58f6c3237ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:04.942967 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f7cc1-fd60-409a-a93f-58f6c3237ef4 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:04.944904 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f7cc1-fd60-409a-a93f-58f6c3237ef4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:04.945911 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f7cc1-fd60-409a-a93f-58f6c3237ef4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:04.951980 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f7cc1-fd60-409a-a93f-58f6c3237ef4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:04.979105 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 468/935] 10.222.0.45 () {56 vars in 1115 bytes} [Mon Aug 1 13:55:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:04.981421 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f7cc1-fd60-409a-a93f-58f6c3237ef4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:05.028142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 468/936] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:55:05.069945 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eeab8c-7696-422d-b47d-062ea66cae2f None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:05.154314 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:05.181360 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f7cc1-fd60-409a-a93f-58f6c3237ef4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:05.235499 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:05.255865 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a9e84-aee0-4ac4-92c1-18a6eeabe615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:05.257522 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a9e84-aee0-4ac4-92c1-18a6eeabe615 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:05.258496 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a9e84-aee0-4ac4-92c1-18a6eeabe615 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:05.259333 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a9e84-aee0-4ac4-92c1-18a6eeabe615 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:05.276838 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eeab8c-7696-422d-b47d-062ea66cae2f None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:05.280821 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19a9e84-aee0-4ac4-92c1-18a6eeabe615 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:05.282978 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19a9e84-aee0-4ac4-92c1-18a6eeabe615 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:05.317638 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-c19a9e84-aee0-4ac4-92c1-18a6eeabe615 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:55:05.317638 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:05.317638 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:05.317638 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:05.317638 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:05.317638 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:05.317638 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:05.317638 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:05.317638 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:05.317638 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:05.317638 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:05.317638 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:05.317638 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:55:05.317638 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:55:05.317638 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:55:05.319720 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:55:05.319720 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:05.319720 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:05.319720 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:05.319720 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:05.319720 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:05.319720 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:05.319720 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:05.319720 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:05.319720 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:05.319720 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:05.319720 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:05.319720 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:05.319720 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:05.319720 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:05.319720 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:05.319720 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:05.320957 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:55:05.320957 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:55:05.320957 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:55:05.320957 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:05.328687 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 469/937] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:55:05.331473 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a9e84-aee0-4ac4-92c1-18a6eeabe615 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:05.363820 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41d1a2-19d4-46cf-890c-0570826b2daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:05.364937 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41d1a2-19d4-46cf-890c-0570826b2daa None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:05.365329 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41d1a2-19d4-46cf-890c-0570826b2daa None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:05.365678 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41d1a2-19d4-46cf-890c-0570826b2daa None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:05.366482 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41d1a2-19d4-46cf-890c-0570826b2daa None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:05.368593 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41d1a2-19d4-46cf-890c-0570826b2daa None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:05.383161 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-2a41d1a2-19d4-46cf-890c-0570826b2daa None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 13:55:05.383161 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:05.383161 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:55:05.383161 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:55:05.383161 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:55:05.383161 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:55:05.383161 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:55:05.383161 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:55:05.383161 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 13:55:05.383161 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:05.383161 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:55:05.383161 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:05.383161 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:05.383161 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:05.383161 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:05.383161 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:05.384312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:05.384312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:05.384312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:05.384312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:05.384312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:05.384312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:05.384312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:05.384312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:55:05.384312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:55:05.384312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:55:05.384312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:55:05.384312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:05.384312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:05.384312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:05.384312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:05.384312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:05.385180 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:05.385180 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:05.385180 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:05.385180 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:05.385180 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:05.385180 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:05.385180 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:05.385180 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:05.385180 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:05.385180 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:05.385180 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:05.385180 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:55:05.385180 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:55:05.385180 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 13:55:05.385180 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:05.385180 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 469/938] 10.222.0.45 () {56 vars in 1132 bytes} [Mon Aug 1 13:55:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:55:05.393152 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41d1a2-19d4-46cf-890c-0570826b2daa None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:05.509146 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a9e84-aee0-4ac4-92c1-18a6eeabe615 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:05.564582 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea247f9-0f75-4e04-8875-1b456c354e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:05.564899 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41d1a2-19d4-46cf-890c-0570826b2daa None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:05.566443 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea247f9-0f75-4e04-8875-1b456c354e4b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:05.567197 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea247f9-0f75-4e04-8875-1b456c354e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:05.567952 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea247f9-0f75-4e04-8875-1b456c354e4b None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:05.569178 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea247f9-0f75-4e04-8875-1b456c354e4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:05.569674 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea247f9-0f75-4e04-8875-1b456c354e4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:05.571885 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea247f9-0f75-4e04-8875-1b456c354e4b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:05.576927 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:05.608899 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 470/939] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:55:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:05.616085 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea247f9-0f75-4e04-8875-1b456c354e4b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:05.646473 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e1500-8b33-41d0-885b-8d3911bd235c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:05.647625 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e1500-8b33-41d0-885b-8d3911bd235c None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:05.648076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e1500-8b33-41d0-885b-8d3911bd235c None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:05.648590 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e1500-8b33-41d0-885b-8d3911bd235c None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:05.649877 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e1500-8b33-41d0-885b-8d3911bd235c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:05.650413 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e1500-8b33-41d0-885b-8d3911bd235c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:05.653527 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e1500-8b33-41d0-885b-8d3911bd235c None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:05.668594 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 470/940] 10.222.0.45 () {56 vars in 1126 bytes} [Mon Aug 1 13:55:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:05.674399 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e1500-8b33-41d0-885b-8d3911bd235c None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:05.691215 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:05.805227 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea247f9-0f75-4e04-8875-1b456c354e4b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:05.865833 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf205c-bf83-44de-adda-30d825d80bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:05.867309 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf205c-bf83-44de-adda-30d825d80bdf None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:05.867852 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf205c-bf83-44de-adda-30d825d80bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:05.868459 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf205c-bf83-44de-adda-30d825d80bdf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:05.886058 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adf205c-bf83-44de-adda-30d825d80bdf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:05.887499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e1500-8b33-41d0-885b-8d3911bd235c None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:05.889070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adf205c-bf83-44de-adda-30d825d80bdf None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:05.906910 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-1adf205c-bf83-44de-adda-30d825d80bdf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 13:55:05.906910 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:05.906910 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:05.906910 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:05.906910 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:05.906910 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:05.906910 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:05.906910 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:05.906910 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:05.906910 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:05.906910 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:05.906910 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:05.906910 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:55:05.906910 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:55:05.906910 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:55:05.906910 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:55:05.907964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:05.907964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:05.907964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:55:05.907964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:55:05.907964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:55:05.907964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:55:05.907964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:05.907964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:05.907964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:05.907964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:05.907964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:05.907964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:05.907964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:05.907964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:05.907964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:05.907964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:05.907964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:05.907964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:05.909353 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:05.909353 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:05.909353 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:55:05.909353 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:55:05.909353 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:55:05.909353 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:55:05.909353 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:55:05.909353 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:55:05.909353 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 13:55:05.909353 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:05.916584 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 471/941] 10.222.0.45 () {56 vars in 1117 bytes} [Mon Aug 1 13:55:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:05.924815 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf205c-bf83-44de-adda-30d825d80bdf None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:05.970928 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd6bf2-1e22-4172-b70e-9295440da607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:05.972077 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd6bf2-1e22-4172-b70e-9295440da607 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:05.972641 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd6bf2-1e22-4172-b70e-9295440da607 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:05.973135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd6bf2-1e22-4172-b70e-9295440da607 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:05.974523 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd6bf2-1e22-4172-b70e-9295440da607 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:05.975070 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd6bf2-1e22-4172-b70e-9295440da607 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:05.979279 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd6bf2-1e22-4172-b70e-9295440da607 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:05.990283 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 471/942] 10.222.0.45 () {56 vars in 1126 bytes} [Mon Aug 1 13:55:05 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:06.000929 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd6bf2-1e22-4172-b70e-9295440da607 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:06.084428 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:06.144850 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf205c-bf83-44de-adda-30d825d80bdf None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:06.149324 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:06.233183 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e942cf-9484-451c-aaa6-634f561d8fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:06.235330 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e942cf-9484-451c-aaa6-634f561d8fce None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:06.236317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e942cf-9484-451c-aaa6-634f561d8fce None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:06.237666 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e942cf-9484-451c-aaa6-634f561d8fce None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:06.240610 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e942cf-9484-451c-aaa6-634f561d8fce None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:06.242021 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e942cf-9484-451c-aaa6-634f561d8fce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:06.246475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e942cf-9484-451c-aaa6-634f561d8fce None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:06.258218 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd6bf2-1e22-4172-b70e-9295440da607 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:06.289499 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 472/943] 10.222.0.45 () {56 vars in 1149 bytes} [Mon Aug 1 13:55:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:06.298709 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e942cf-9484-451c-aaa6-634f561d8fce None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:06.335255 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c2bf3-4602-4d6a-b938-8b704a5b7fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:06.337165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c2bf3-4602-4d6a-b938-8b704a5b7fdb None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:06.338306 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c2bf3-4602-4d6a-b938-8b704a5b7fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:06.339170 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c2bf3-4602-4d6a-b938-8b704a5b7fdb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:06.341190 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c2bf3-4602-4d6a-b938-8b704a5b7fdb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:06.344616 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c2bf3-4602-4d6a-b938-8b704a5b7fdb None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:06.390195 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 472/944] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 13:55:05 2022] POST /identity/v3/endpoints => generated 347 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 13:55:06.481570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e942cf-9484-451c-aaa6-634f561d8fce None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:06.557983 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51739071-033f-47bd-8e9d-a17f145c3f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:06.559428 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51739071-033f-47bd-8e9d-a17f145c3f2e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:06.560123 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51739071-033f-47bd-8e9d-a17f145c3f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:06.560761 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51739071-033f-47bd-8e9d-a17f145c3f2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:06.583423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51739071-033f-47bd-8e9d-a17f145c3f2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:06.586729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51739071-033f-47bd-8e9d-a17f145c3f2e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:06.610341 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-51739071-033f-47bd-8e9d-a17f145c3f2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:55:06.610341 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:06.610341 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:06.610341 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:06.610341 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:06.610341 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:06.610341 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:06.610341 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:06.610341 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:06.610341 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:06.610341 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:06.610341 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:06.610341 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:55:06.610341 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:55:06.610341 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:55:06.611384 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:55:06.611384 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:06.611384 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:06.611384 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:06.611384 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:06.611384 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:06.611384 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:06.611384 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:06.611384 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:06.611384 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:06.611384 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:06.611384 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:06.611384 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:06.611384 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:06.611384 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:06.611384 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:06.611384 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:06.612358 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:55:06.612358 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:55:06.612358 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:55:06.612358 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:06.619912 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 473/945] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:55:06.622564 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c2bf3-4602-4d6a-b938-8b704a5b7fdb None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:06.796396 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:06.825769 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c2bf3-4602-4d6a-b938-8b704a5b7fdb None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:06.893969 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ab0a6-c5ce-45f6-baa5-351bf8997a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:06.896140 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ab0a6-c5ce-45f6-baa5-351bf8997a29 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:06.896784 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ab0a6-c5ce-45f6-baa5-351bf8997a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:06.897283 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ab0a6-c5ce-45f6-baa5-351bf8997a29 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:06.898328 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ab0a6-c5ce-45f6-baa5-351bf8997a29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:06.898889 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ab0a6-c5ce-45f6-baa5-351bf8997a29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:06.900620 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ab0a6-c5ce-45f6-baa5-351bf8997a29 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:06.920403 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 473/946] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:06.924321 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51739071-033f-47bd-8e9d-a17f145c3f2e None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:06.939920 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:07.059312 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51739071-033f-47bd-8e9d-a17f145c3f2e None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:07.099393 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7342b73-def5-48e4-a161-93ea5674c626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:07.100528 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7342b73-def5-48e4-a161-93ea5674c626 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:07.101017 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7342b73-def5-48e4-a161-93ea5674c626 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:07.101514 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7342b73-def5-48e4-a161-93ea5674c626 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:07.111251 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7342b73-def5-48e4-a161-93ea5674c626 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:07.114446 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7342b73-def5-48e4-a161-93ea5674c626 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:07.129211 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-c7342b73-def5-48e4-a161-93ea5674c626 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:55:07.129211 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:07.129211 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:07.129211 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:07.129211 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:07.129211 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:07.129211 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:07.129211 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:07.129211 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:07.129211 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:07.129211 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:07.129211 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:07.129211 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:55:07.129211 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:55:07.129211 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:55:07.130891 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:55:07.130891 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:07.130891 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:07.130891 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:07.130891 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:07.130891 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:07.130891 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:07.130891 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:07.130891 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:07.130891 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:07.130891 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:07.130891 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:07.130891 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:07.130891 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:07.130891 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:07.130891 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:07.130891 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:07.132619 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:55:07.132619 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:55:07.132619 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:55:07.132619 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:55:07.132619 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:55:07.132619 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:55:07.132619 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:55:07.132619 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:07.136376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 474/947] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:55:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:07.148512 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ab0a6-c5ce-45f6-baa5-351bf8997a29 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:07.320611 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ab0a6-c5ce-45f6-baa5-351bf8997a29 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:07.368748 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c9ae22-7467-401f-9edb-7ca936891fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:07.370223 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9ae22-7467-401f-9edb-7ca936891fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:07.370947 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9ae22-7467-401f-9edb-7ca936891fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:07.371669 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9ae22-7467-401f-9edb-7ca936891fb3 None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:07.373235 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c9ae22-7467-401f-9edb-7ca936891fb3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:07.374044 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c9ae22-7467-401f-9edb-7ca936891fb3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:07.376899 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c9ae22-7467-401f-9edb-7ca936891fb3 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:07.404399 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 474/948] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:55:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:07.407003 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7342b73-def5-48e4-a161-93ea5674c626 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:07.500320 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:07.556388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7342b73-def5-48e4-a161-93ea5674c626 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:07.599772 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f200353-bd66-45be-8d61-a2a7c789f3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:07.604382 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f200353-bd66-45be-8d61-a2a7c789f3d4 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:07.604382 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f200353-bd66-45be-8d61-a2a7c789f3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:07.604382 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f200353-bd66-45be-8d61-a2a7c789f3d4 None admin] PATH_INFO: `/v3/projects/36f063d8f65d494a89dbbd72c53417c8/users/e07e83c0bdba47d39c09308eba1bdaae/roles/c29b2bae1d6c4e438cb861ed195d5c72` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:07.604382 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f200353-bd66-45be-8d61-a2a7c789f3d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=36f063d8f65d494a89dbbd72c53417c8, user_id=e07e83c0bdba47d39c09308eba1bdaae, role_id=c29b2bae1d6c4e438cb861ed195d5c72)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:07.604937 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f200353-bd66-45be-8d61-a2a7c789f3d4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:07.640350 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 475/949] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 13:55:07 2022] PUT /identity/v3/projects/36f063d8f65d494a89dbbd72c53417c8/users/e07e83c0bdba47d39c09308eba1bdaae/roles/c29b2bae1d6c4e438cb861ed195d5c72 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:55:07.784520 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb86d3-5ad2-44a6-b32b-b7d5450395f2 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:07.784762 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb86d3-5ad2-44a6-b32b-b7d5450395f2 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:07.785055 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb86d3-5ad2-44a6-b32b-b7d5450395f2 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:07.786728 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 475/950] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:55:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:55:07.797232 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79780d-34d1-4138-badb-4c76dda87b0f None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:07.797499 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79780d-34d1-4138-badb-4c76dda87b0f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:07.797816 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79780d-34d1-4138-badb-4c76dda87b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:07.821954 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-ac79780d-34d1-4138-badb-4c76dda87b0f None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:07.823686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ac79780d-34d1-4138-badb-4c76dda87b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:07.913823 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 476/951] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:55:07.956860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeb86d3-5ad2-44a6-b32b-b7d5450395f2 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:07.982817 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:08.070983 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcfc55-6ceb-4d43-922b-57c79bbb3e56 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:08.071801 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcfc55-6ceb-4d43-922b-57c79bbb3e56 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:08.071941 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcfc55-6ceb-4d43-922b-57c79bbb3e56 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:08.083578 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 477/952] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:55:08 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:55:08.091182 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8818aa-1c60-47b6-a4b6-724337b5002b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:08.091182 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8818aa-1c60-47b6-a4b6-724337b5002b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:08.091182 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8818aa-1c60-47b6-a4b6-724337b5002b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:08.125987 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-fe8818aa-1c60-47b6-a4b6-724337b5002b None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:08.128384 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fe8818aa-1c60-47b6-a4b6-724337b5002b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:08.241413 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeb86d3-5ad2-44a6-b32b-b7d5450395f2 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:08.256097 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:08.292934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 478/953] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:55:08.329287 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8818aa-1c60-47b6-a4b6-724337b5002b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:08.358252 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b427fe-c91c-4964-b9e8-7514c266050e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:08.360190 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b427fe-c91c-4964-b9e8-7514c266050e None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:08.361070 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b427fe-c91c-4964-b9e8-7514c266050e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:08.361864 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b427fe-c91c-4964-b9e8-7514c266050e None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:08.363653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b427fe-c91c-4964-b9e8-7514c266050e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:08.367024 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b427fe-c91c-4964-b9e8-7514c266050e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:08.371993 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: WARNING keystone.server.flask.common [None req-23b427fe-c91c-4964-b9e8-7514c266050e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 01 13:55:08.397328 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.sql.core [None req-23b427fe-c91c-4964-b9e8-7514c266050e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 13:55:08.397328 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 01 13:55:08.397328 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [parameters: {'user_id': '740e8f8f9ab74f889ef79154ad500393', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 01 13:55:08.397328 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80310) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 13:55:08.412029 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-23b427fe-c91c-4964-b9e8-7514c266050e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 01 13:55:08.412029 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:08.412029 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 13:55:08.412029 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 13:55:08.412029 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 13:55:08.412029 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 13:55:08.412029 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 13:55:08.412029 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 13:55:08.412029 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 13:55:08.412029 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application conn.query(q) Aug 01 13:55:08.412029 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 13:55:08.412029 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 13:55:08.412029 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 13:55:08.412029 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application result.read() Aug 01 13:55:08.412029 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 13:55:08.413857 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 13:55:08.413857 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 13:55:08.413857 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 13:55:08.413857 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 13:55:08.413857 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 13:55:08.413857 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 13:55:08.413857 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 13:55:08.413857 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 13:55:08.413857 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:08.413857 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 13:55:08.413857 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:08.413857 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:08.413857 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 13:55:08.413857 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 13:55:08.413857 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 01 13:55:08.413857 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 01 13:55:08.413857 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 13:55:08.413857 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application next(self.gen) Aug 01 13:55:08.415428 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 13:55:08.415428 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application yield resource Aug 01 13:55:08.415428 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 13:55:08.415428 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application next(self.gen) Aug 01 13:55:08.415428 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 13:55:08.415428 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 13:55:08.415428 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 13:55:08.415428 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 13:55:08.415428 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 13:55:08.415428 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise self.value Aug 01 13:55:08.415428 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 13:55:08.415428 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 13:55:08.415428 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 13:55:08.415428 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application session.commit() Aug 01 13:55:08.415428 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 13:55:08.415428 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 13:55:08.415428 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 13:55:08.418859 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 13:55:08.418859 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 13:55:08.418859 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self.session.flush() Aug 01 13:55:08.418859 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 13:55:08.418859 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 13:55:08.418859 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 13:55:08.418859 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 13:55:08.418859 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 13:55:08.418859 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application compat.raise_( Aug 01 13:55:08.418859 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 13:55:08.418859 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception Aug 01 13:55:08.418859 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 13:55:08.418859 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 13:55:08.418859 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 13:55:08.418859 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 13:55:08.418859 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 13:55:08.418859 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 13:55:08.422505 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 13:55:08.422505 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 13:55:08.422505 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 01 13:55:08.422505 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 01 13:55:08.422505 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 13:55:08.422505 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 13:55:08.422505 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 13:55:08.422505 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 13:55:08.422505 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 13:55:08.422505 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 13:55:08.422505 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 13:55:08.422505 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 13:55:08.422505 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 13:55:08.422505 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 13:55:08.422505 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 13:55:08.422505 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception Aug 01 13:55:08.425614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 13:55:08.425614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 13:55:08.425614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 13:55:08.425614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 13:55:08.425614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 13:55:08.425614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 13:55:08.425614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 13:55:08.425614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application conn.query(q) Aug 01 13:55:08.425614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 13:55:08.425614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 13:55:08.425614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 13:55:08.425614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application result.read() Aug 01 13:55:08.425614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 13:55:08.425614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 13:55:08.425614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 13:55:08.425614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 13:55:08.429037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 13:55:08.429037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 13:55:08.429037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 13:55:08.429037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 13:55:08.429037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 13:55:08.429037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 01 13:55:08.429037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [parameters: {'user_id': '740e8f8f9ab74f889ef79154ad500393', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 01 13:55:08.429037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 13:55:08.429037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:08.429037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:55:08.429037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:08.429037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:08.429037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:08.429037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:08.429037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:08.432559 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:08.432559 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:08.432559 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:08.432559 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:08.432559 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:08.432559 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:08.432559 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:08.432559 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 01 13:55:08.432559 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 01 13:55:08.432559 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:08.432559 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:08.432559 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:55:08.432559 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:55:08.432559 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:55:08.432559 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:55:08.432559 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 01 13:55:08.432559 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 01 13:55:08.438680 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 01 13:55:08.438680 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 01 13:55:08.438680 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 13:55:08.438680 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 13:55:08.438680 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 01 13:55:08.438680 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:08.438680 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 476/954] 10.222.0.45 () {60 vars in 1131 bytes} [Mon Aug 1 13:55:07 2022] POST /identity/v3/users => generated 160 bytes in 458 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:55:08.438680 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b427fe-c91c-4964-b9e8-7514c266050e None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:08.558314 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8818aa-1c60-47b6-a4b6-724337b5002b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:08.582604 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:08.624141 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114f484-07f7-4af1-ae9b-06f6ac377748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:08.625409 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114f484-07f7-4af1-ae9b-06f6ac377748 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:08.625914 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114f484-07f7-4af1-ae9b-06f6ac377748 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:08.626420 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114f484-07f7-4af1-ae9b-06f6ac377748 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:08.627753 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c114f484-07f7-4af1-ae9b-06f6ac377748 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:08.630280 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c114f484-07f7-4af1-ae9b-06f6ac377748 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:08.648312 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-c114f484-07f7-4af1-ae9b-06f6ac377748 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 13:55:08.648312 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:08.648312 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:55:08.648312 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:55:08.648312 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:55:08.648312 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:55:08.648312 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:55:08.648312 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:55:08.648312 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 13:55:08.648312 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:08.648312 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:55:08.648312 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:08.648312 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:08.648312 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:08.648312 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:08.648312 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:08.648312 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:08.650466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:08.650466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:08.650466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:08.650466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:08.650466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:08.650466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:08.650466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:55:08.650466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:55:08.650466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:55:08.650466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:55:08.650466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:08.650466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:08.650466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:08.650466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:08.650466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:08.650466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:08.652199 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:08.652199 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:08.652199 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:08.652199 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:08.652199 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:08.652199 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:08.652199 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:08.652199 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:08.652199 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:08.652199 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:08.652199 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:55:08.652199 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:55:08.652199 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 13:55:08.652199 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:08.656265 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 479/955] 10.222.0.45 () {56 vars in 1126 bytes} [Mon Aug 1 13:55:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:55:08.657640 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b427fe-c91c-4964-b9e8-7514c266050e None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:08.661975 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114f484-07f7-4af1-ae9b-06f6ac377748 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:08.729194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73927d64-6afd-4005-83c5-e79ce09b0418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:08.730265 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73927d64-6afd-4005-83c5-e79ce09b0418 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:08.730742 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73927d64-6afd-4005-83c5-e79ce09b0418 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:08.731193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73927d64-6afd-4005-83c5-e79ce09b0418 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:08.743313 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73927d64-6afd-4005-83c5-e79ce09b0418 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:08.746806 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73927d64-6afd-4005-83c5-e79ce09b0418 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:08.762312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-73927d64-6afd-4005-83c5-e79ce09b0418 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 13:55:08.762312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:08.762312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:08.762312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:08.762312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:08.762312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:08.762312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:08.762312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:08.762312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:08.762312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:08.762312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:08.762312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:08.762312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:55:08.762312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:55:08.762312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:55:08.762312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:55:08.763906 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:08.763906 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:08.763906 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:55:08.763906 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:55:08.763906 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:55:08.763906 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:55:08.763906 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:08.763906 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:08.763906 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:08.763906 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:08.763906 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:08.763906 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:08.763906 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:08.763906 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:08.763906 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:08.763906 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:08.763906 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:08.763906 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:08.765624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:08.765624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:08.765624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:55:08.765624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:55:08.765624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:55:08.765624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:55:08.765624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:55:08.765624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:55:08.765624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 13:55:08.765624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:08.769358 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:08.770351 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 477/956] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:55:08.778714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73927d64-6afd-4005-83c5-e79ce09b0418 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:08.877785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114f484-07f7-4af1-ae9b-06f6ac377748 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:08.950130 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e72ec8-d84f-4f43-a36a-d9b49324484c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:08.951162 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e72ec8-d84f-4f43-a36a-d9b49324484c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:08.951640 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e72ec8-d84f-4f43-a36a-d9b49324484c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:08.952104 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e72ec8-d84f-4f43-a36a-d9b49324484c None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:08.953394 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e72ec8-d84f-4f43-a36a-d9b49324484c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:08.953906 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e72ec8-d84f-4f43-a36a-d9b49324484c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:08.956455 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e72ec8-d84f-4f43-a36a-d9b49324484c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:08.972391 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 480/957] 10.222.0.45 () {56 vars in 1122 bytes} [Mon Aug 1 13:55:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:08.977443 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73927d64-6afd-4005-83c5-e79ce09b0418 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:08.985595 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e72ec8-d84f-4f43-a36a-d9b49324484c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:09.027399 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:09.038980 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1ee0b-2599-4bff-8986-94887c4f42d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:09.040302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1ee0b-2599-4bff-8986-94887c4f42d0 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:09.040791 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1ee0b-2599-4bff-8986-94887c4f42d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:09.041234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1ee0b-2599-4bff-8986-94887c4f42d0 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:09.042193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d1ee0b-2599-4bff-8986-94887c4f42d0 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:09.042879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d1ee0b-2599-4bff-8986-94887c4f42d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:09.045094 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d1ee0b-2599-4bff-8986-94887c4f42d0 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:09.106466 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 478/958] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 13:55:08 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:09.224998 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e72ec8-d84f-4f43-a36a-d9b49324484c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:09.284834 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:09.300217 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78adb978-c36e-4dfb-9884-d0c12382cf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:09.301986 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78adb978-c36e-4dfb-9884-d0c12382cf0d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:09.303088 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78adb978-c36e-4dfb-9884-d0c12382cf0d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:09.304430 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78adb978-c36e-4dfb-9884-d0c12382cf0d None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:09.306136 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78adb978-c36e-4dfb-9884-d0c12382cf0d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:09.307117 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78adb978-c36e-4dfb-9884-d0c12382cf0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:09.310213 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78adb978-c36e-4dfb-9884-d0c12382cf0d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:09.321399 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 481/959] 10.222.0.45 () {56 vars in 1122 bytes} [Mon Aug 1 13:55:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:09.334140 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1ee0b-2599-4bff-8986-94887c4f42d0 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:09.539828 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1ee0b-2599-4bff-8986-94887c4f42d0 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:09.610505 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e35f8-23dc-45d6-80eb-ff8bb6ac357a None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:09.610987 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e35f8-23dc-45d6-80eb-ff8bb6ac357a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:09.611867 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e35f8-23dc-45d6-80eb-ff8bb6ac357a None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:09.613757 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 482/960] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:55:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:09.620350 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc947b-0a70-4617-a6c7-620442af1855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:09.621424 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc947b-0a70-4617-a6c7-620442af1855 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:09.621853 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc947b-0a70-4617-a6c7-620442af1855 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:09.623001 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc947b-0a70-4617-a6c7-620442af1855 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:09.623299 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5e034-8a52-414d-b5c2-6ba580dd77d3 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:09.623678 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc947b-0a70-4617-a6c7-620442af1855 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:09.623955 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5e034-8a52-414d-b5c2-6ba580dd77d3 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:09.624254 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5e034-8a52-414d-b5c2-6ba580dd77d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:09.625599 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc947b-0a70-4617-a6c7-620442af1855 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:09.658027 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-1bb5e034-8a52-414d-b5c2-6ba580dd77d3 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:09.668720 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 479/961] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 13:55:09 2022] POST /identity/v3/endpoints => generated 349 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:55:09.669017 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1bb5e034-8a52-414d-b5c2-6ba580dd77d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:09.818263 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 483/962] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:55:09.856554 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc947b-0a70-4617-a6c7-620442af1855 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:09.943968 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:10.075713 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc947b-0a70-4617-a6c7-620442af1855 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:10.149586 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a87625-c224-4440-8d98-32d29c78e698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:10.150730 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a87625-c224-4440-8d98-32d29c78e698 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:10.151237 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a87625-c224-4440-8d98-32d29c78e698 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:10.151825 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a87625-c224-4440-8d98-32d29c78e698 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:10.153087 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a87625-c224-4440-8d98-32d29c78e698 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:10.155744 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a87625-c224-4440-8d98-32d29c78e698 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:10.174734 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-10a87625-c224-4440-8d98-32d29c78e698 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 13:55:10.174734 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:10.174734 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:55:10.174734 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:55:10.174734 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:55:10.174734 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:55:10.174734 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:55:10.174734 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:55:10.174734 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 13:55:10.174734 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:10.174734 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:55:10.174734 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:10.174734 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:10.174734 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:10.174734 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:10.174734 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:10.174734 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:10.175985 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:10.175985 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:10.175985 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:10.175985 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:10.175985 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:10.175985 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:10.175985 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:55:10.175985 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:55:10.175985 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:55:10.175985 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:55:10.175985 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:10.175985 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:10.175985 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:10.175985 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:10.175985 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:10.175985 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:10.182393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:10.182393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:10.182393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:10.182393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:10.182393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:10.182393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:10.182393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:10.182393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:10.182393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:10.182393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:10.182393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:55:10.182393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:55:10.182393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 13:55:10.182393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:10.182393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 480/963] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:55:09 2022] GET /identity/v3/services/compute => generated 88 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:10.185653 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb5e034-8a52-414d-b5c2-6ba580dd77d3 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:10.388874 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb5e034-8a52-414d-b5c2-6ba580dd77d3 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:10.405237 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:10.463542 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740786d3-06c2-4cbc-b9ed-0f0129d76717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:10.464641 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740786d3-06c2-4cbc-b9ed-0f0129d76717 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:10.465089 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740786d3-06c2-4cbc-b9ed-0f0129d76717 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:10.465529 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740786d3-06c2-4cbc-b9ed-0f0129d76717 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:10.466769 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740786d3-06c2-4cbc-b9ed-0f0129d76717 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:10.467261 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740786d3-06c2-4cbc-b9ed-0f0129d76717 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:10.469496 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740786d3-06c2-4cbc-b9ed-0f0129d76717 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:10.478850 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 484/964] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Aug 1 13:55:10 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:10.487556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a87625-c224-4440-8d98-32d29c78e698 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:10.750041 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a87625-c224-4440-8d98-32d29c78e698 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:10.773063 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:10.826125 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dbdcf6-67f8-49cc-bad3-a573706586b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:10.827277 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbdcf6-67f8-49cc-bad3-a573706586b8 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:10.827763 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbdcf6-67f8-49cc-bad3-a573706586b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:10.828509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbdcf6-67f8-49cc-bad3-a573706586b8 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:10.829797 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dbdcf6-67f8-49cc-bad3-a573706586b8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:10.830345 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dbdcf6-67f8-49cc-bad3-a573706586b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:10.832996 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dbdcf6-67f8-49cc-bad3-a573706586b8 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:10.848788 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 481/965] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Aug 1 13:55:10 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:10.920525 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e61eaf-9af3-4e33-b08e-c256cac8c365 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:10.921036 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e61eaf-9af3-4e33-b08e-c256cac8c365 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:10.921566 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e61eaf-9af3-4e33-b08e-c256cac8c365 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:10.924464 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 485/966] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:55:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:10.931544 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e105477-7ead-437b-b410-97b31926c832 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:10.931841 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e105477-7ead-437b-b410-97b31926c832 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:10.932203 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e105477-7ead-437b-b410-97b31926c832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:10.953201 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-2e105477-7ead-437b-b410-97b31926c832 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:10.954885 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-2e105477-7ead-437b-b410-97b31926c832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:11.075460 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 482/967] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:55:11.117034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e61eaf-9af3-4e33-b08e-c256cac8c365 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:11.285361 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e61eaf-9af3-4e33-b08e-c256cac8c365 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:11.350960 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0e821-aa25-4e06-8f21-2116363060fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:11.351827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0e821-aa25-4e06-8f21-2116363060fa None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:11.352319 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0e821-aa25-4e06-8f21-2116363060fa None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:11.352780 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0e821-aa25-4e06-8f21-2116363060fa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:11.371727 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa0e821-aa25-4e06-8f21-2116363060fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:11.380123 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-efa0e821-aa25-4e06-8f21-2116363060fa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 13:55:11.380123 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:11.380123 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:11.380123 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:11.380123 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:11.380123 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:11.380123 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:11.380123 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:11.380123 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:11.380123 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:11.380123 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:11.380123 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:11.380123 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:11.380123 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:11.380123 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:11.380123 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:11.382029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:11.382029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:11.382029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:11.382029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:11.382029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:11.382029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:11.382029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:11.382029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:11.382029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:11.382029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:11.382029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:11.382029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:11.382029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:11.382029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:11.382029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:11.382029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:11.383691 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:11.383691 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:11.383691 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:11.383691 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:11.383691 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:11.383691 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:11.383691 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:11.383691 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:11.383691 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:11.383691 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:11.383691 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 13:55:11.383691 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:11.383691 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 486/968] 10.222.0.45 () {56 vars in 1120 bytes} [Mon Aug 1 13:55:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:11.394121 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e105477-7ead-437b-b410-97b31926c832 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:11.620872 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e105477-7ead-437b-b410-97b31926c832 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:11.626261 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:11.693116 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76219622-e554-4548-a350-37d3957c63a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:11.694221 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76219622-e554-4548-a350-37d3957c63a2 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:11.694725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76219622-e554-4548-a350-37d3957c63a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:11.695224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76219622-e554-4548-a350-37d3957c63a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:11.696621 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76219622-e554-4548-a350-37d3957c63a2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:11.697180 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76219622-e554-4548-a350-37d3957c63a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:11.699884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76219622-e554-4548-a350-37d3957c63a2 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:11.724389 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 483/969] 10.222.0.45 () {56 vars in 1115 bytes} [Mon Aug 1 13:55:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:11.726489 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0e821-aa25-4e06-8f21-2116363060fa None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:11.789677 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:11.927246 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0e821-aa25-4e06-8f21-2116363060fa None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:11.991951 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd864e5-7500-4a20-be91-f828b4fb413c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:11.993480 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd864e5-7500-4a20-be91-f828b4fb413c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:11.994199 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd864e5-7500-4a20-be91-f828b4fb413c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:11.994964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd864e5-7500-4a20-be91-f828b4fb413c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:12.015315 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd864e5-7500-4a20-be91-f828b4fb413c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:12.018409 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd864e5-7500-4a20-be91-f828b4fb413c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:12.055050 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-5cd864e5-7500-4a20-be91-f828b4fb413c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:55:12.055050 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:12.055050 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:12.055050 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:12.055050 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:12.055050 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:12.055050 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:12.055050 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:12.055050 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:12.055050 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:12.055050 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:12.055050 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:12.055050 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:55:12.055050 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:55:12.055050 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:55:12.056963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:55:12.056963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:12.056963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:12.056963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:12.056963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:12.056963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:12.056963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:12.056963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:12.056963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:12.056963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:12.056963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:12.056963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:12.056963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:12.056963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:12.056963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:12.056963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:12.056963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:12.058824 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:55:12.058824 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:55:12.058824 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:55:12.058824 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:12.058824 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 487/970] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:12.070094 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76219622-e554-4548-a350-37d3957c63a2 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:12.264008 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76219622-e554-4548-a350-37d3957c63a2 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:12.334161 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64160e5-b25a-4e4e-892e-4a05c09b5846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:12.335560 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64160e5-b25a-4e4e-892e-4a05c09b5846 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:12.336243 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64160e5-b25a-4e4e-892e-4a05c09b5846 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:12.336955 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64160e5-b25a-4e4e-892e-4a05c09b5846 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:12.338483 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64160e5-b25a-4e4e-892e-4a05c09b5846 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:12.339222 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64160e5-b25a-4e4e-892e-4a05c09b5846 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:12.341960 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64160e5-b25a-4e4e-892e-4a05c09b5846 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:12.366381 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 484/971] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:55:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:12.375563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd864e5-7500-4a20-be91-f828b4fb413c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:12.537679 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:12.571936 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd864e5-7500-4a20-be91-f828b4fb413c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:12.637999 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8263331b-e3a6-47ef-bb94-ac6b909515fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:12.638974 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263331b-e3a6-47ef-bb94-ac6b909515fb None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:12.639395 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263331b-e3a6-47ef-bb94-ac6b909515fb None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:12.639812 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263331b-e3a6-47ef-bb94-ac6b909515fb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:12.655427 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8263331b-e3a6-47ef-bb94-ac6b909515fb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:12.658265 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8263331b-e3a6-47ef-bb94-ac6b909515fb None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:12.681304 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-8263331b-e3a6-47ef-bb94-ac6b909515fb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 13:55:12.681304 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:12.681304 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:12.681304 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:12.681304 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:12.681304 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:12.681304 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:12.681304 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:12.681304 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:12.681304 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:12.681304 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:12.681304 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:12.681304 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:55:12.681304 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:55:12.681304 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:55:12.681304 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:55:12.683270 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:12.683270 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:12.683270 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:55:12.683270 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:55:12.683270 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:55:12.683270 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:55:12.683270 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:12.683270 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:12.683270 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:12.683270 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:12.683270 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:12.683270 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:12.683270 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:12.683270 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:12.683270 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:12.683270 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:12.683270 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:12.683270 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:12.685822 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:12.685822 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:12.685822 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:55:12.685822 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:55:12.685822 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:55:12.685822 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:55:12.685822 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:55:12.685822 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:55:12.685822 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 13:55:12.685822 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:12.687490 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 488/972] 10.222.0.45 () {56 vars in 1117 bytes} [Mon Aug 1 13:55:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:12.692970 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64160e5-b25a-4e4e-892e-4a05c09b5846 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:12.746894 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:12.853380 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c9967-4097-42a4-a40a-4fb0edc15691 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:12.854110 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c9967-4097-42a4-a40a-4fb0edc15691 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:12.854260 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c9967-4097-42a4-a40a-4fb0edc15691 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:12.857032 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 489/973] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:55:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:55:12.866161 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe3580-78fa-405e-8afa-78fcd148187a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:12.866646 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe3580-78fa-405e-8afa-78fcd148187a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:12.867105 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe3580-78fa-405e-8afa-78fcd148187a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:12.897607 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64160e5-b25a-4e4e-892e-4a05c09b5846 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:12.899727 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-cebe3580-78fa-405e-8afa-78fcd148187a None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:12.902164 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-cebe3580-78fa-405e-8afa-78fcd148187a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:12.965564 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06543fbb-9b37-4911-9a0b-fa4390eea956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:12.966649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06543fbb-9b37-4911-9a0b-fa4390eea956 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:12.967214 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06543fbb-9b37-4911-9a0b-fa4390eea956 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:12.967619 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06543fbb-9b37-4911-9a0b-fa4390eea956 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:12.969232 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06543fbb-9b37-4911-9a0b-fa4390eea956 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:12.969770 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06543fbb-9b37-4911-9a0b-fa4390eea956 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:12.972708 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06543fbb-9b37-4911-9a0b-fa4390eea956 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:13.009691 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 485/974] 10.222.0.45 () {56 vars in 1149 bytes} [Mon Aug 1 13:55:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:13.019054 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06543fbb-9b37-4911-9a0b-fa4390eea956 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:13.037869 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 490/975] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:55:13.075612 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe3580-78fa-405e-8afa-78fcd148187a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:13.198061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:13.213366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:13.242227 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06543fbb-9b37-4911-9a0b-fa4390eea956 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:13.279280 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe3580-78fa-405e-8afa-78fcd148187a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:13.306072 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b27bd2-76a8-45f4-aa69-60f0f0e9ec65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:13.306711 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b27bd2-76a8-45f4-aa69-60f0f0e9ec65 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:13.306983 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b27bd2-76a8-45f4-aa69-60f0f0e9ec65 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:13.307249 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b27bd2-76a8-45f4-aa69-60f0f0e9ec65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:13.347601 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff21bf-5520-49b2-bcdc-51ca2f13b806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:13.348765 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff21bf-5520-49b2-bcdc-51ca2f13b806 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:13.349273 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff21bf-5520-49b2-bcdc-51ca2f13b806 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:13.349769 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff21bf-5520-49b2-bcdc-51ca2f13b806 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:13.350963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff21bf-5520-49b2-bcdc-51ca2f13b806 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:13.353682 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff21bf-5520-49b2-bcdc-51ca2f13b806 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:13.371593 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-1aff21bf-5520-49b2-bcdc-51ca2f13b806 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 13:55:13.371593 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:13.371593 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:55:13.371593 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:55:13.371593 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:55:13.371593 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:55:13.371593 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:55:13.371593 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:55:13.371593 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 13:55:13.371593 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:13.371593 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:55:13.371593 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:13.371593 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:13.371593 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:13.371593 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:13.371593 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:13.371593 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:13.373631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:13.373631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:13.373631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:13.373631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:13.373631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:13.373631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:13.373631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:55:13.373631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:55:13.373631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:55:13.373631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:55:13.373631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:13.373631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:13.373631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:13.373631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:13.373631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:13.373631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:13.375456 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:13.375456 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:13.375456 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:13.375456 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:13.375456 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:13.375456 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:13.375456 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:13.375456 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:13.375456 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:13.375456 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:13.375456 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:55:13.375456 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:55:13.375456 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 13:55:13.375456 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:13.380693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 491/976] 10.222.0.45 () {56 vars in 1126 bytes} [Mon Aug 1 13:55:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:55:13.387051 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff21bf-5520-49b2-bcdc-51ca2f13b806 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:13.421548 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b27bd2-76a8-45f4-aa69-60f0f0e9ec65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:13.424511 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b27bd2-76a8-45f4-aa69-60f0f0e9ec65 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:13.594437 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff21bf-5520-49b2-bcdc-51ca2f13b806 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:13.612399 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:13.641481 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 486/977] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:13 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:55:13.643802 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48b31a-3c14-4360-9160-8e1149607073 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:13.644310 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48b31a-3c14-4360-9160-8e1149607073 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:13.644811 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48b31a-3c14-4360-9160-8e1149607073 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:13.647484 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 487/978] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:55:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:13.652110 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48b31a-3c14-4360-9160-8e1149607073 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:13.666666 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c36b1-f7e0-46e6-96d0-b65f4c834016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:13.667502 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c36b1-f7e0-46e6-96d0-b65f4c834016 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:13.668566 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c36b1-f7e0-46e6-96d0-b65f4c834016 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:13.668979 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c36b1-f7e0-46e6-96d0-b65f4c834016 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:13.669819 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c36b1-f7e0-46e6-96d0-b65f4c834016 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:13.670243 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c36b1-f7e0-46e6-96d0-b65f4c834016 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:13.672110 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c36b1-f7e0-46e6-96d0-b65f4c834016 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:13.674498 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:13.685913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 492/979] 10.222.0.45 () {56 vars in 1122 bytes} [Mon Aug 1 13:55:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:13.686956 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3335c87-96b3-4b45-b017-f772d85bcec1 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:13.687482 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3335c87-96b3-4b45-b017-f772d85bcec1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:13.688202 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3335c87-96b3-4b45-b017-f772d85bcec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:13.711223 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b3335c87-96b3-4b45-b017-f772d85bcec1 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:13.713379 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b3335c87-96b3-4b45-b017-f772d85bcec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:13.820629 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 493/980] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:55:13.820629 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3335c87-96b3-4b45-b017-f772d85bcec1 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:13.825541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48b31a-3c14-4360-9160-8e1149607073 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:13.870919 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb322e-0f5f-466d-8772-a56a6c65f827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:13.871973 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb322e-0f5f-466d-8772-a56a6c65f827 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:13.872699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb322e-0f5f-466d-8772-a56a6c65f827 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:13.873100 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb322e-0f5f-466d-8772-a56a6c65f827 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:13.892458 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccb322e-0f5f-466d-8772-a56a6c65f827 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:13.895173 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccb322e-0f5f-466d-8772-a56a6c65f827 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:13.922523 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-2ccb322e-0f5f-466d-8772-a56a6c65f827 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:55:13.922523 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:13.922523 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:13.922523 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:13.922523 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:13.922523 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:13.922523 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:13.922523 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:13.922523 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:13.922523 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:13.922523 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:13.922523 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:13.922523 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:55:13.922523 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:55:13.922523 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:55:13.924143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:55:13.924143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:13.924143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:13.924143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:13.924143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:13.924143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:13.924143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:13.924143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:13.924143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:13.924143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:13.924143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:13.924143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:13.924143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:13.924143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:13.924143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:13.924143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:13.924143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:13.925727 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:55:13.925727 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:55:13.925727 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:55:13.925727 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:13.925727 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 488/981] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:55:13.926253 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb322e-0f5f-466d-8772-a56a6c65f827 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:13.995251 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:13.996485 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3335c87-96b3-4b45-b017-f772d85bcec1 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:14.040662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:14.050738 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa695487-c146-4815-abbb-2f555d385f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:14.051588 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa695487-c146-4815-abbb-2f555d385f51 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:14.052122 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa695487-c146-4815-abbb-2f555d385f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:14.052618 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa695487-c146-4815-abbb-2f555d385f51 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:14.053893 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa695487-c146-4815-abbb-2f555d385f51 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:14.054237 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa695487-c146-4815-abbb-2f555d385f51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:14.055652 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa695487-c146-4815-abbb-2f555d385f51 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:14.061389 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 494/982] 10.222.0.45 () {56 vars in 1122 bytes} [Mon Aug 1 13:55:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:14.062250 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa695487-c146-4815-abbb-2f555d385f51 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:14.094466 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb322e-0f5f-466d-8772-a56a6c65f827 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:14.156601 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6106c1e-b0df-4b5f-90e3-84c7a4f0b341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:14.157601 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6106c1e-b0df-4b5f-90e3-84c7a4f0b341 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:14.158011 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6106c1e-b0df-4b5f-90e3-84c7a4f0b341 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:14.158436 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6106c1e-b0df-4b5f-90e3-84c7a4f0b341 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:14.159440 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6106c1e-b0df-4b5f-90e3-84c7a4f0b341 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:14.161159 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6106c1e-b0df-4b5f-90e3-84c7a4f0b341 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:14.185654 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 489/983] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Aug 1 13:55:13 2022] POST /identity/v3/services => generated 242 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:55:14.186886 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6106c1e-b0df-4b5f-90e3-84c7a4f0b341 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:14.210620 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa695487-c146-4815-abbb-2f555d385f51 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:14.287098 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf908628-b4d3-436d-b04b-23e1bef470e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:14.288197 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf908628-b4d3-436d-b04b-23e1bef470e8 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:14.288633 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf908628-b4d3-436d-b04b-23e1bef470e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:14.289090 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf908628-b4d3-436d-b04b-23e1bef470e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:14.290461 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf908628-b4d3-436d-b04b-23e1bef470e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:14.291002 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf908628-b4d3-436d-b04b-23e1bef470e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:14.293750 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf908628-b4d3-436d-b04b-23e1bef470e8 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:14.316176 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 495/984] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:55:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:14.325791 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf908628-b4d3-436d-b04b-23e1bef470e8 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:14.406348 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:14.417795 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:14.418758 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6106c1e-b0df-4b5f-90e3-84c7a4f0b341 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:14.480530 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d929136-a07f-46e4-bf86-0bd88dc4d7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:14.481755 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d929136-a07f-46e4-bf86-0bd88dc4d7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:14.482244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d929136-a07f-46e4-bf86-0bd88dc4d7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:14.483243 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d929136-a07f-46e4-bf86-0bd88dc4d7b4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:14.485179 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d929136-a07f-46e4-bf86-0bd88dc4d7b4 None admin] RBAC: Adding query filter params (interface=admin, service_id=0e1e13fec9324c1bacd27cdae5829d49, region_id=RegionOne) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:14.485981 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d929136-a07f-46e4-bf86-0bd88dc4d7b4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:14.488979 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d929136-a07f-46e4-bf86-0bd88dc4d7b4 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:14.499146 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 490/985] 10.222.0.45 () {56 vars in 1249 bytes} [Mon Aug 1 13:55:14 2022] GET /identity/v3/endpoints?service_id=0e1e13fec9324c1bacd27cdae5829d49&interface=admin®ion_id=RegionOne => generated 193 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:14.507801 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d929136-a07f-46e4-bf86-0bd88dc4d7b4 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:14.515280 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf908628-b4d3-436d-b04b-23e1bef470e8 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:14.584563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad7e91-320a-400d-939a-6eb85c18fc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:14.585569 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad7e91-320a-400d-939a-6eb85c18fc54 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:14.586003 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad7e91-320a-400d-939a-6eb85c18fc54 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:14.586425 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad7e91-320a-400d-939a-6eb85c18fc54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:14.597334 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad7e91-320a-400d-939a-6eb85c18fc54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:14.598487 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad7e91-320a-400d-939a-6eb85c18fc54 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:14.614668 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-bbad7e91-320a-400d-939a-6eb85c18fc54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:55:14.614668 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:14.614668 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:14.614668 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:14.614668 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:14.614668 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:14.614668 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:14.614668 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:14.614668 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:14.614668 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:14.614668 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:14.614668 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:14.614668 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:55:14.614668 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:55:14.614668 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:55:14.615934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:55:14.615934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:14.615934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:14.615934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:14.615934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:14.615934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:14.615934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:14.615934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:14.615934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:14.615934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:14.615934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:14.615934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:14.615934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:14.615934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:14.615934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:14.615934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:14.615934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:14.617034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:55:14.617034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:55:14.617034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:55:14.617034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:55:14.617034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:55:14.617034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:55:14.617034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:55:14.617034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:14.617034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 496/986] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:55:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:14.622171 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad7e91-320a-400d-939a-6eb85c18fc54 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:14.682749 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d929136-a07f-46e4-bf86-0bd88dc4d7b4 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:14.742453 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d4614-72c5-4fa0-a23c-7e24ce1e3815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:14.743512 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d4614-72c5-4fa0-a23c-7e24ce1e3815 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:14.743965 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d4614-72c5-4fa0-a23c-7e24ce1e3815 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:14.744446 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d4614-72c5-4fa0-a23c-7e24ce1e3815 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:14.745574 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d4614-72c5-4fa0-a23c-7e24ce1e3815 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:14.748250 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d4614-72c5-4fa0-a23c-7e24ce1e3815 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:14.763066 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 491/987] 10.222.0.45 () {56 vars in 1087 bytes} [Mon Aug 1 13:55:14 2022] GET /identity/v3/services => generated 1727 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 13:55:14.777423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad7e91-320a-400d-939a-6eb85c18fc54 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:14.795714 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:14.840242 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a068a4-6380-4033-8aaa-286ea5e6c28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:14.842138 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a068a4-6380-4033-8aaa-286ea5e6c28a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:14.842786 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a068a4-6380-4033-8aaa-286ea5e6c28a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:14.843552 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a068a4-6380-4033-8aaa-286ea5e6c28a None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:14.845413 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a068a4-6380-4033-8aaa-286ea5e6c28a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:14.846238 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a068a4-6380-4033-8aaa-286ea5e6c28a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:14.849376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a068a4-6380-4033-8aaa-286ea5e6c28a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:14.873050 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 497/988] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:55:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:14.881554 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d4614-72c5-4fa0-a23c-7e24ce1e3815 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:14.918692 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:15.053892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d4614-72c5-4fa0-a23c-7e24ce1e3815 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:15.107117 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee177b-0973-453e-aabb-a0afaa5332dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:15.107942 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee177b-0973-453e-aabb-a0afaa5332dc None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:15.108620 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee177b-0973-453e-aabb-a0afaa5332dc None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:15.108994 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee177b-0973-453e-aabb-a0afaa5332dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:15.110051 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ee177b-0973-453e-aabb-a0afaa5332dc None admin] RBAC: Adding query filter params (role.id=c29b2bae1d6c4e438cb861ed195d5c72, scope.project.id=36f063d8f65d494a89dbbd72c53417c8, user.id=e07e83c0bdba47d39c09308eba1bdaae) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:15.110495 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ee177b-0973-453e-aabb-a0afaa5332dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:15.112163 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ee177b-0973-453e-aabb-a0afaa5332dc None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:15.128307 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 492/989] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:55:14 2022] GET /identity/v3/role_assignments?user.id=e07e83c0bdba47d39c09308eba1bdaae&scope.project.id=36f063d8f65d494a89dbbd72c53417c8&role.id=c29b2bae1d6c4e438cb861ed195d5c72 => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:17.191179 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc044290-d680-41af-82ca-3a11e0682a0c None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:17.191930 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc044290-d680-41af-82ca-3a11e0682a0c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:17.191930 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc044290-d680-41af-82ca-3a11e0682a0c None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:17.193192 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 498/990] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:55:17.200135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb25a98-4bc0-4a90-b485-00c93f991c63 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:17.201023 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb25a98-4bc0-4a90-b485-00c93f991c63 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:17.201179 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb25a98-4bc0-4a90-b485-00c93f991c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:17.225529 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-feb25a98-4bc0-4a90-b485-00c93f991c63 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:17.227275 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-feb25a98-4bc0-4a90-b485-00c93f991c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:17.324353 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 493/991] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:17 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Aug 01 13:55:17.361920 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc044290-d680-41af-82ca-3a11e0682a0c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:17.546493 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc044290-d680-41af-82ca-3a11e0682a0c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:17.574693 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290b45d-3f37-4d77-9a6d-d94e6517fe81 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:17.575110 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290b45d-3f37-4d77-9a6d-d94e6517fe81 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:17.575882 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290b45d-3f37-4d77-9a6d-d94e6517fe81 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:17.578154 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 494/992] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:55:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:55:17.585246 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ccc84-e9fd-40c7-a30c-09fad6741e0e None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:17.585515 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ccc84-e9fd-40c7-a30c-09fad6741e0e None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:17.585883 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ccc84-e9fd-40c7-a30c-09fad6741e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:17.611243 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42461b66-0a23-4a37-9487-430735daddeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:17.611592 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-190ccc84-e9fd-40c7-a30c-09fad6741e0e None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:17.612248 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42461b66-0a23-4a37-9487-430735daddeb None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:17.612717 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42461b66-0a23-4a37-9487-430735daddeb None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:17.613234 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42461b66-0a23-4a37-9487-430735daddeb None admin] PATH_INFO: `/v3/services/compute` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:17.613405 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-190ccc84-e9fd-40c7-a30c-09fad6741e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:17.614216 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42461b66-0a23-4a37-9487-430735daddeb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:17.616442 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42461b66-0a23-4a37-9487-430735daddeb None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:17.631827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-42461b66-0a23-4a37-9487-430735daddeb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 13:55:17.631827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:17.631827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:55:17.631827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:55:17.631827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:55:17.631827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:55:17.631827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:55:17.631827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:55:17.631827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 13:55:17.631827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:17.631827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:55:17.631827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:17.631827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:17.631827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:17.631827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:17.631827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:17.631827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:17.633411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:17.633411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:17.633411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:17.633411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:17.633411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:17.633411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:17.633411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:55:17.633411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:55:17.633411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:55:17.633411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:55:17.633411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:17.633411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:17.633411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:17.633411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:17.633411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:17.633411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:17.634663 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:17.634663 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:17.634663 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:17.634663 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:17.634663 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:17.634663 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:17.634663 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:17.634663 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:17.634663 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:17.634663 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:17.634663 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:55:17.634663 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:55:17.634663 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 13:55:17.634663 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:17.634663 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 499/993] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:55:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:17.639090 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:17.640346 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42461b66-0a23-4a37-9487-430735daddeb None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:17.714391 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:17.725318 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 495/994] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:17 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:55:17.762580 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ccc84-e9fd-40c7-a30c-09fad6741e0e None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:17.847345 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42461b66-0a23-4a37-9487-430735daddeb None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:17.912590 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ce5be-3223-411c-8d45-a8f3663e03f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:17.913655 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ce5be-3223-411c-8d45-a8f3663e03f6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:17.914141 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ce5be-3223-411c-8d45-a8f3663e03f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:17.914633 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ce5be-3223-411c-8d45-a8f3663e03f6 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:17.915789 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127ce5be-3223-411c-8d45-a8f3663e03f6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:17.916364 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127ce5be-3223-411c-8d45-a8f3663e03f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:17.918615 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127ce5be-3223-411c-8d45-a8f3663e03f6 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:17.919314 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ccc84-e9fd-40c7-a30c-09fad6741e0e None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:17.927872 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 500/995] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Aug 1 13:55:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:17.930138 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6766b37-3e77-4900-848b-b4370cf40dc2 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:17.930583 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6766b37-3e77-4900-848b-b4370cf40dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:17.931036 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6766b37-3e77-4900-848b-b4370cf40dc2 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:17.933657 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 501/996] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:55:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:17.937962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6766b37-3e77-4900-848b-b4370cf40dc2 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:17.983642 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f1d87a-d751-4ab6-a198-3fcdd5cae3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:17.985287 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1d87a-d751-4ab6-a198-3fcdd5cae3dd None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:17.985774 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1d87a-d751-4ab6-a198-3fcdd5cae3dd None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:17.986268 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1d87a-d751-4ab6-a198-3fcdd5cae3dd None admin] PATH_INFO: `/v3/services/image` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:17.986952 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f1d87a-d751-4ab6-a198-3fcdd5cae3dd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:17.989661 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f1d87a-d751-4ab6-a198-3fcdd5cae3dd None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:18.004500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-82f1d87a-d751-4ab6-a198-3fcdd5cae3dd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 13:55:18.004500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:18.004500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:55:18.004500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:55:18.004500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:55:18.004500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:55:18.004500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:55:18.004500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:55:18.004500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 13:55:18.004500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:18.004500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:55:18.004500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:18.004500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:18.004500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:18.004500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:18.004500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:18.004500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:18.006083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:18.006083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:18.006083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:18.006083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:18.006083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:18.006083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:18.006083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:55:18.006083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:55:18.006083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:55:18.006083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:55:18.006083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:18.006083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:18.006083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:18.006083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:18.006083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:18.006083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:18.007782 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:18.007782 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:18.007782 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:18.007782 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:18.007782 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:18.007782 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:18.007782 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:18.007782 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:18.007782 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:18.007782 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:18.007782 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:55:18.007782 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:55:18.007782 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 13:55:18.007782 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:18.007782 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 496/997] 10.222.0.45 () {56 vars in 1105 bytes} [Mon Aug 1 13:55:17 2022] GET /identity/v3/services/image => generated 86 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:55:18.009480 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482d427-c18e-4b3e-abab-103fe66a5ff8 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:18.009480 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482d427-c18e-4b3e-abab-103fe66a5ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:18.009480 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482d427-c18e-4b3e-abab-103fe66a5ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:18.035767 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-3482d427-c18e-4b3e-abab-103fe66a5ff8 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:18.037907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3482d427-c18e-4b3e-abab-103fe66a5ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:18.052572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:18.117411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:18.127464 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6766b37-3e77-4900-848b-b4370cf40dc2 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:18.162598 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 497/998] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:18 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:55:18.163683 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3482d427-c18e-4b3e-abab-103fe66a5ff8 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:18.192095 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4a472-e7d3-469e-bf98-528aa1b68d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:18.193183 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4a472-e7d3-469e-bf98-528aa1b68d4f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:18.193667 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4a472-e7d3-469e-bf98-528aa1b68d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:18.194130 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4a472-e7d3-469e-bf98-528aa1b68d4f None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:18.195346 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4a472-e7d3-469e-bf98-528aa1b68d4f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:18.195889 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4a472-e7d3-469e-bf98-528aa1b68d4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:18.198503 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4a472-e7d3-469e-bf98-528aa1b68d4f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:18.205339 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 502/999] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Aug 1 13:55:17 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:18.209370 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4a472-e7d3-469e-bf98-528aa1b68d4f None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:18.289129 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3482d427-c18e-4b3e-abab-103fe66a5ff8 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:18.330059 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606839d-7864-4622-8f7e-445ab82109cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:18.330874 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606839d-7864-4622-8f7e-445ab82109cb None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:18.331224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606839d-7864-4622-8f7e-445ab82109cb None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:18.331546 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606839d-7864-4622-8f7e-445ab82109cb None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:18.332488 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606839d-7864-4622-8f7e-445ab82109cb None admin] RBAC: Adding query filter params (name=image) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:18.332842 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606839d-7864-4622-8f7e-445ab82109cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:18.334301 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606839d-7864-4622-8f7e-445ab82109cb None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:18.340488 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 498/1000] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:18.341445 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606839d-7864-4622-8f7e-445ab82109cb None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:18.345046 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4a472-e7d3-469e-bf98-528aa1b68d4f None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:18.352688 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:18.387619 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb9b59-f31f-4835-b712-6a70dc45a693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:18.388581 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb9b59-f31f-4835-b712-6a70dc45a693 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:18.389018 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb9b59-f31f-4835-b712-6a70dc45a693 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:18.389474 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb9b59-f31f-4835-b712-6a70dc45a693 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:18.390590 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb9b59-f31f-4835-b712-6a70dc45a693 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:18.392849 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb9b59-f31f-4835-b712-6a70dc45a693 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:18.409260 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-2bcb9b59-f31f-4835-b712-6a70dc45a693 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 13:55:18.409260 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:18.409260 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:55:18.409260 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:55:18.409260 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:55:18.409260 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:55:18.409260 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:55:18.409260 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:55:18.409260 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 13:55:18.409260 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:18.409260 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:55:18.409260 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:18.409260 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:18.409260 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:18.409260 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:18.409260 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:18.409260 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:18.410594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:18.410594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:18.410594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:18.410594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:18.410594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:18.410594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:18.410594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:55:18.410594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:55:18.410594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:55:18.410594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:55:18.410594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:18.410594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:18.410594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:18.410594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:18.410594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:18.410594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:18.411605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:18.411605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:18.411605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:18.411605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:18.411605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:18.411605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:18.411605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:18.411605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:18.411605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:18.411605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:18.411605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:55:18.411605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:55:18.411605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 13:55:18.411605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:18.412422 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 503/1001] 10.222.0.45 () {56 vars in 1126 bytes} [Mon Aug 1 13:55:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:18.413465 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb9b59-f31f-4835-b712-6a70dc45a693 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:18.461994 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:18.486149 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606839d-7864-4622-8f7e-445ab82109cb None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:18.537808 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ff8aa-95e6-4174-b741-6fd49d9e2542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:18.538902 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ff8aa-95e6-4174-b741-6fd49d9e2542 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:18.539312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ff8aa-95e6-4174-b741-6fd49d9e2542 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:18.539642 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ff8aa-95e6-4174-b741-6fd49d9e2542 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:18.540491 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ff8aa-95e6-4174-b741-6fd49d9e2542 None admin] RBAC: Adding query filter params (interface=public, service_id=86a719e3dc04414f90cbffe0ddbddd49, region_id=RegionOne) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:18.540807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ff8aa-95e6-4174-b741-6fd49d9e2542 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:18.542271 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ff8aa-95e6-4174-b741-6fd49d9e2542 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:18.548539 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 499/1002] 10.222.0.45 () {56 vars in 1251 bytes} [Mon Aug 1 13:55:18 2022] GET /identity/v3/endpoints?service_id=86a719e3dc04414f90cbffe0ddbddd49&interface=public®ion_id=RegionOne => generated 194 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:18.549200 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ff8aa-95e6-4174-b741-6fd49d9e2542 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:18.587941 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb9b59-f31f-4835-b712-6a70dc45a693 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:18.627806 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:18.642123 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1424155-1e42-4133-8d7e-82858a9aea01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:18.642681 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1424155-1e42-4133-8d7e-82858a9aea01 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:18.642963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1424155-1e42-4133-8d7e-82858a9aea01 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:18.643235 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1424155-1e42-4133-8d7e-82858a9aea01 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:18.643926 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1424155-1e42-4133-8d7e-82858a9aea01 None admin] RBAC: Adding query filter params (type=image) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:18.644230 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1424155-1e42-4133-8d7e-82858a9aea01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:18.645647 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1424155-1e42-4133-8d7e-82858a9aea01 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:18.651909 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 504/1003] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:18 2022] GET /identity/v3/services?type=image => generated 122 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:18.653270 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1424155-1e42-4133-8d7e-82858a9aea01 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:18.690521 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ff8aa-95e6-4174-b741-6fd49d9e2542 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:18.751652 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa75cde0-3d54-4314-9f13-8fde19aa561c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:18.752704 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75cde0-3d54-4314-9f13-8fde19aa561c None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:18.753154 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75cde0-3d54-4314-9f13-8fde19aa561c None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:18.753665 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75cde0-3d54-4314-9f13-8fde19aa561c None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:18.754835 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa75cde0-3d54-4314-9f13-8fde19aa561c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:18.755339 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa75cde0-3d54-4314-9f13-8fde19aa561c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:18.758126 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa75cde0-3d54-4314-9f13-8fde19aa561c None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:18.767536 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 500/1004] 10.222.0.45 () {56 vars in 1122 bytes} [Mon Aug 1 13:55:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:18.777171 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa75cde0-3d54-4314-9f13-8fde19aa561c None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:18.833677 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:18.851930 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1424155-1e42-4133-8d7e-82858a9aea01 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:18.933651 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e1884-dd68-49f7-be38-1b8508651ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:18.934680 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e1884-dd68-49f7-be38-1b8508651ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:18.935176 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e1884-dd68-49f7-be38-1b8508651ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:18.935668 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e1884-dd68-49f7-be38-1b8508651ad5 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:18.937037 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e1884-dd68-49f7-be38-1b8508651ad5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:18.939757 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e1884-dd68-49f7-be38-1b8508651ad5 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:18.960322 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 505/1005] 10.222.0.45 () {56 vars in 1087 bytes} [Mon Aug 1 13:55:18 2022] GET /identity/v3/services => generated 1727 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 13:55:19.017505 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa75cde0-3d54-4314-9f13-8fde19aa561c None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:19.118265 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c24f12-6877-43a7-a319-4930064ff71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:19.119452 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c24f12-6877-43a7-a319-4930064ff71f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:19.120026 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c24f12-6877-43a7-a319-4930064ff71f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:19.120541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c24f12-6877-43a7-a319-4930064ff71f None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:19.121819 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c24f12-6877-43a7-a319-4930064ff71f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:19.130935 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c24f12-6877-43a7-a319-4930064ff71f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:19.130935 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c24f12-6877-43a7-a319-4930064ff71f None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:19.130935 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:19.145054 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 501/1006] 10.222.0.45 () {56 vars in 1122 bytes} [Mon Aug 1 13:55:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:19.147558 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e1884-dd68-49f7-be38-1b8508651ad5 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:19.350796 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e1884-dd68-49f7-be38-1b8508651ad5 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:19.424029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d9027-92b9-4561-869d-b3db48f9a652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:19.425006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d9027-92b9-4561-869d-b3db48f9a652 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:19.425409 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d9027-92b9-4561-869d-b3db48f9a652 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:19.425804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d9027-92b9-4561-869d-b3db48f9a652 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:19.426947 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76d9027-92b9-4561-869d-b3db48f9a652 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:19.429369 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76d9027-92b9-4561-869d-b3db48f9a652 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:19.468314 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 506/1007] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 13:55:19 2022] POST /identity/v3/endpoints => generated 325 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 13:55:21.315665 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a67e3-809c-4d5b-ae90-0ae4bae7e9ac None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:21.316907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a67e3-809c-4d5b-ae90-0ae4bae7e9ac None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:21.316907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a67e3-809c-4d5b-ae90-0ae4bae7e9ac None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:21.319417 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 502/1008] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:55:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:55:21.326395 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f659a-9b8a-4a72-b47f-c2a9683e665a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:21.327452 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f659a-9b8a-4a72-b47f-c2a9683e665a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:21.327452 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f659a-9b8a-4a72-b47f-c2a9683e665a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:21.352437 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-279f659a-9b8a-4a72-b47f-c2a9683e665a None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:21.353936 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-279f659a-9b8a-4a72-b47f-c2a9683e665a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:21.373562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:21.519049 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 507/1009] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:21 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:55:21.550500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1a67e3-809c-4d5b-ae90-0ae4bae7e9ac None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:21.748642 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1a67e3-809c-4d5b-ae90-0ae4bae7e9ac None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:21.795430 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d741c44c-108d-40b2-b38b-06624e05ba72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:21.796854 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741c44c-108d-40b2-b38b-06624e05ba72 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:21.797429 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741c44c-108d-40b2-b38b-06624e05ba72 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:21.797949 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741c44c-108d-40b2-b38b-06624e05ba72 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:21.799152 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d741c44c-108d-40b2-b38b-06624e05ba72 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:21.801224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d741c44c-108d-40b2-b38b-06624e05ba72 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:21.836379 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 503/1010] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Aug 1 13:55:21 2022] POST /identity/v3/services => generated 242 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:55:21.949655 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8459f005-6688-4d9a-8e0f-2d832b751533 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:21.950162 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8459f005-6688-4d9a-8e0f-2d832b751533 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:21.950484 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8459f005-6688-4d9a-8e0f-2d832b751533 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:21.952791 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 508/1011] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:55:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:55:21.963434 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2052fd2-923e-45c9-a5e0-861df4bb0fa3 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:21.963985 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2052fd2-923e-45c9-a5e0-861df4bb0fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:21.964375 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2052fd2-923e-45c9-a5e0-861df4bb0fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:22.005411 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-a2052fd2-923e-45c9-a5e0-861df4bb0fa3 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:22.007600 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a2052fd2-923e-45c9-a5e0-861df4bb0fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:22.149046 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:22.208338 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 504/1012] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:21 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:55:22.247732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8459f005-6688-4d9a-8e0f-2d832b751533 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:22.439148 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07f2d1-dcee-43bb-943f-ee47369c40a2 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:22.439911 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07f2d1-dcee-43bb-943f-ee47369c40a2 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:22.440061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07f2d1-dcee-43bb-943f-ee47369c40a2 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:22.442530 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 505/1013] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:55:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:55:22.451727 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47291a3b-ce9a-44f6-9a5a-2b477d88f48b None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:22.452455 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47291a3b-ce9a-44f6-9a5a-2b477d88f48b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:22.452940 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47291a3b-ce9a-44f6-9a5a-2b477d88f48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:22.457903 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8459f005-6688-4d9a-8e0f-2d832b751533 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:22.482712 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-47291a3b-ce9a-44f6-9a5a-2b477d88f48b None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:22.484661 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-47291a3b-ce9a-44f6-9a5a-2b477d88f48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:22.541656 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c1b9e-2faa-4027-9fb6-4bd93b9b448c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:22.542608 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c1b9e-2faa-4027-9fb6-4bd93b9b448c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:22.542927 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c1b9e-2faa-4027-9fb6-4bd93b9b448c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:22.543364 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c1b9e-2faa-4027-9fb6-4bd93b9b448c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:22.545834 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568c1b9e-2faa-4027-9fb6-4bd93b9b448c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:22.547869 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568c1b9e-2faa-4027-9fb6-4bd93b9b448c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:22.552426 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 509/1014] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:22.560356 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c1b9e-2faa-4027-9fb6-4bd93b9b448c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:22.589318 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:22.633040 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 506/1015] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:22 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:55:22.670990 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47291a3b-ce9a-44f6-9a5a-2b477d88f48b None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:22.797293 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c1b9e-2faa-4027-9fb6-4bd93b9b448c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:22.863004 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47291a3b-ce9a-44f6-9a5a-2b477d88f48b None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:22.873328 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec967abe-195a-44c6-b27a-29e341df9516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:22.874477 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec967abe-195a-44c6-b27a-29e341df9516 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:22.875037 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec967abe-195a-44c6-b27a-29e341df9516 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:22.875583 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec967abe-195a-44c6-b27a-29e341df9516 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:22.877011 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec967abe-195a-44c6-b27a-29e341df9516 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:22.880063 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec967abe-195a-44c6-b27a-29e341df9516 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:22.889984 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:22.936099 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c02dc-f4c1-4a38-9ddf-019e14a724b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:22.937244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c02dc-f4c1-4a38-9ddf-019e14a724b7 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:22.937794 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c02dc-f4c1-4a38-9ddf-019e14a724b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:22.938224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c02dc-f4c1-4a38-9ddf-019e14a724b7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:22.939165 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 510/1016] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Aug 1 13:55:22 2022] POST /identity/v3/projects => generated 311 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:55:22.939527 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c02dc-f4c1-4a38-9ddf-019e14a724b7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:22.942161 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c02dc-f4c1-4a38-9ddf-019e14a724b7 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:22.959192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-b74c02dc-f4c1-4a38-9ddf-019e14a724b7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 13:55:22.959192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:22.959192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:55:22.959192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:55:22.959192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:55:22.959192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:55:22.959192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:55:22.959192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:55:22.959192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 13:55:22.959192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:22.959192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:55:22.959192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:22.959192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:22.959192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:22.959192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:22.959192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:22.959192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:22.961557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:22.961557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:22.961557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:22.961557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:22.961557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:22.961557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:22.961557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:55:22.961557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:55:22.961557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:55:22.961557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:55:22.961557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:22.961557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:22.961557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:22.961557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:22.961557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:22.961557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:22.963541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:22.963541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:22.963541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:22.963541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:22.963541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:22.963541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:22.963541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:22.963541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:22.963541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:22.963541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:22.963541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:55:22.963541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:55:22.963541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 13:55:22.963541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:22.967580 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 507/1017] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:55:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:55:22.971806 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec967abe-195a-44c6-b27a-29e341df9516 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:23.107450 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:23.205995 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec967abe-195a-44c6-b27a-29e341df9516 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:23.269457 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c5f041-9ca2-4a91-ba1c-a6344531c44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:23.270150 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5f041-9ca2-4a91-ba1c-a6344531c44b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:23.270488 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5f041-9ca2-4a91-ba1c-a6344531c44b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:23.270769 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5f041-9ca2-4a91-ba1c-a6344531c44b None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:23.271847 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c5f041-9ca2-4a91-ba1c-a6344531c44b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:23.272158 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c5f041-9ca2-4a91-ba1c-a6344531c44b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:23.273733 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c5f041-9ca2-4a91-ba1c-a6344531c44b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:23.280191 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 511/1018] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Aug 1 13:55:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:23.287362 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c02dc-f4c1-4a38-9ddf-019e14a724b7 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:23.508061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c02dc-f4c1-4a38-9ddf-019e14a724b7 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:23.581163 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef697b8-1e67-478c-8065-dbbba81edd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:23.582602 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef697b8-1e67-478c-8065-dbbba81edd99 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:23.583357 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef697b8-1e67-478c-8065-dbbba81edd99 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:23.584054 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef697b8-1e67-478c-8065-dbbba81edd99 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:23.587797 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef697b8-1e67-478c-8065-dbbba81edd99 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:23.588696 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef697b8-1e67-478c-8065-dbbba81edd99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:23.592455 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef697b8-1e67-478c-8065-dbbba81edd99 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:23.606323 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 508/1019] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Aug 1 13:55:23 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:23.625867 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c5f041-9ca2-4a91-ba1c-a6344531c44b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:23.823352 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c5f041-9ca2-4a91-ba1c-a6344531c44b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:23.872199 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:23.889602 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842d0af-57e4-4837-8708-e5003ae9ca62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:23.890410 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842d0af-57e4-4837-8708-e5003ae9ca62 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:23.892099 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842d0af-57e4-4837-8708-e5003ae9ca62 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:23.893768 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842d0af-57e4-4837-8708-e5003ae9ca62 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:23.894690 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842d0af-57e4-4837-8708-e5003ae9ca62 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:23.896730 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842d0af-57e4-4837-8708-e5003ae9ca62 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:23.932767 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 512/1020] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 13:55:23 2022] POST /identity/v3/endpoints => generated 334 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:55:24.657134 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f9e1ee-5105-46b0-b801-efdbb8ab6a2c None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:24.657959 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f9e1ee-5105-46b0-b801-efdbb8ab6a2c None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:24.657959 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f9e1ee-5105-46b0-b801-efdbb8ab6a2c None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:24.659586 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 509/1021] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:24.665313 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875ba05-d087-4ea6-88d8-fe4d20e49a54 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:24.666082 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875ba05-d087-4ea6-88d8-fe4d20e49a54 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:24.666082 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875ba05-d087-4ea6-88d8-fe4d20e49a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:24.689252 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-6875ba05-d087-4ea6-88d8-fe4d20e49a54 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:24.690722 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6875ba05-d087-4ea6-88d8-fe4d20e49a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:24.811087 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 513/1022] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:24 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:55:24.836939 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f9e1ee-5105-46b0-b801-efdbb8ab6a2c None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:24.902364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:24.977049 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f9e1ee-5105-46b0-b801-efdbb8ab6a2c None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:25.032988 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea699b45-889f-4d74-9007-2e15b1efe3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:25.034106 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea699b45-889f-4d74-9007-2e15b1efe3e8 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:25.034566 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea699b45-889f-4d74-9007-2e15b1efe3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:25.035049 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea699b45-889f-4d74-9007-2e15b1efe3e8 None admin] PATH_INFO: `/v3/services/image` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:25.036228 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea699b45-889f-4d74-9007-2e15b1efe3e8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:25.038617 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea699b45-889f-4d74-9007-2e15b1efe3e8 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:25.055834 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-ea699b45-889f-4d74-9007-2e15b1efe3e8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 13:55:25.055834 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:25.055834 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:55:25.055834 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:55:25.055834 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:55:25.055834 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:55:25.055834 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:55:25.055834 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:55:25.055834 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 13:55:25.055834 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:25.055834 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:55:25.055834 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:25.055834 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:25.055834 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:25.055834 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:25.055834 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:25.055834 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:25.057192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:25.057192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:25.057192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:25.057192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:25.057192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:25.057192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:25.057192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:55:25.057192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:55:25.057192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:55:25.057192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:55:25.057192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:25.057192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:25.057192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:25.057192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:25.057192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:25.057192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:25.058154 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:25.058154 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:25.058154 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:25.058154 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:25.058154 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:25.058154 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:25.058154 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:25.058154 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:25.058154 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:25.058154 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:25.058154 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:55:25.058154 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:55:25.058154 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 13:55:25.058154 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:25.060675 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 510/1023] 10.222.0.45 () {56 vars in 1105 bytes} [Mon Aug 1 13:55:24 2022] GET /identity/v3/services/image => generated 86 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:55:25.067974 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6875ba05-d087-4ea6-88d8-fe4d20e49a54 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:25.186746 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6875ba05-d087-4ea6-88d8-fe4d20e49a54 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:25.196652 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:25.234753 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a6371-91a1-4240-9cdd-f6f8c352c671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:25.235623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a6371-91a1-4240-9cdd-f6f8c352c671 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:25.235918 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a6371-91a1-4240-9cdd-f6f8c352c671 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:25.236320 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a6371-91a1-4240-9cdd-f6f8c352c671 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:25.237116 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a6371-91a1-4240-9cdd-f6f8c352c671 None admin] RBAC: Adding query filter params (name=image) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:25.237419 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a6371-91a1-4240-9cdd-f6f8c352c671 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:25.238811 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a6371-91a1-4240-9cdd-f6f8c352c671 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:25.249547 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 514/1024] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:25.251471 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea699b45-889f-4d74-9007-2e15b1efe3e8 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:25.400687 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea699b45-889f-4d74-9007-2e15b1efe3e8 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:25.412178 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:25.457214 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8230e451-5c67-458b-819b-8d84f39ed965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:25.457921 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230e451-5c67-458b-819b-8d84f39ed965 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:25.458196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230e451-5c67-458b-819b-8d84f39ed965 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:25.458469 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230e451-5c67-458b-819b-8d84f39ed965 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:25.459163 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8230e451-5c67-458b-819b-8d84f39ed965 None admin] RBAC: Adding query filter params (type=image) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:25.459466 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8230e451-5c67-458b-819b-8d84f39ed965 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:25.460895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8230e451-5c67-458b-819b-8d84f39ed965 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:25.477043 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 511/1025] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:25 2022] GET /identity/v3/services?type=image => generated 350 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:25.478730 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a6371-91a1-4240-9cdd-f6f8c352c671 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:25.660104 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a6371-91a1-4240-9cdd-f6f8c352c671 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:25.709372 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e5db9-7e2a-4e2c-8c81-61ab453e8f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:25.710327 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e5db9-7e2a-4e2c-8c81-61ab453e8f37 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:25.710327 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e5db9-7e2a-4e2c-8c81-61ab453e8f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:25.710584 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e5db9-7e2a-4e2c-8c81-61ab453e8f37 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:25.711500 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690e5db9-7e2a-4e2c-8c81-61ab453e8f37 None admin] RBAC: Adding query filter params (interface=public, service_id=8935bea28aa343548fcb076e2c985fa3, region_id=RegionOne) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:25.712031 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690e5db9-7e2a-4e2c-8c81-61ab453e8f37 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:25.713550 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690e5db9-7e2a-4e2c-8c81-61ab453e8f37 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:25.721459 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 515/1026] 10.222.0.45 () {56 vars in 1251 bytes} [Mon Aug 1 13:55:25 2022] GET /identity/v3/endpoints?service_id=8935bea28aa343548fcb076e2c985fa3&interface=public®ion_id=RegionOne => generated 194 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:25.728897 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8230e451-5c67-458b-819b-8d84f39ed965 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:25.829990 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23611415-8ec6-4c0f-8479-c1cc8a042b3c None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:25.830487 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23611415-8ec6-4c0f-8479-c1cc8a042b3c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:25.830801 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23611415-8ec6-4c0f-8479-c1cc8a042b3c None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:25.833262 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 516/1027] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:55:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:55:25.840751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d9802-8894-4add-8291-94a77e917897 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:25.841245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d9802-8894-4add-8291-94a77e917897 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:25.841741 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d9802-8894-4add-8291-94a77e917897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:25.869280 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-865d9802-8894-4add-8291-94a77e917897 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:25.871190 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-865d9802-8894-4add-8291-94a77e917897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:25.911045 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8230e451-5c67-458b-819b-8d84f39ed965 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:25.920996 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:25.979630 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36a93f7-5771-4ed8-9093-1cfcbcb0b228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:25.980487 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a93f7-5771-4ed8-9093-1cfcbcb0b228 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:25.980800 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a93f7-5771-4ed8-9093-1cfcbcb0b228 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:25.981262 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a93f7-5771-4ed8-9093-1cfcbcb0b228 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:25.982003 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36a93f7-5771-4ed8-9093-1cfcbcb0b228 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:25.983626 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36a93f7-5771-4ed8-9093-1cfcbcb0b228 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:25.992841 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 512/1028] 10.222.0.45 () {56 vars in 1087 bytes} [Mon Aug 1 13:55:25 2022] GET /identity/v3/services => generated 1957 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 13:55:26.005011 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 517/1029] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:55:26.042865 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36a93f7-5771-4ed8-9093-1cfcbcb0b228 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:26.115638 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:26.242949 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36a93f7-5771-4ed8-9093-1cfcbcb0b228 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:26.289926 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff93c65-cc15-4e6c-ac8e-a2a55b937ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:26.290790 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff93c65-cc15-4e6c-ac8e-a2a55b937ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:26.291197 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff93c65-cc15-4e6c-ac8e-a2a55b937ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:26.291599 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff93c65-cc15-4e6c-ac8e-a2a55b937ec5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:26.293147 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff93c65-cc15-4e6c-ac8e-a2a55b937ec5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:26.294832 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff93c65-cc15-4e6c-ac8e-a2a55b937ec5 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:26.297198 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 513/1030] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:26.303453 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d9802-8894-4add-8291-94a77e917897 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:26.462620 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d9802-8894-4add-8291-94a77e917897 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:26.530696 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab7950-4245-4f4d-8856-81c225cefe93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:26.531708 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab7950-4245-4f4d-8856-81c225cefe93 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:26.532202 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab7950-4245-4f4d-8856-81c225cefe93 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:26.532705 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab7950-4245-4f4d-8856-81c225cefe93 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:26.536717 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab7950-4245-4f4d-8856-81c225cefe93 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:26.536717 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab7950-4245-4f4d-8856-81c225cefe93 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:26.580328 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 518/1031] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 13:55:26 2022] POST /identity/v3/users => generated 285 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 13:55:26.792831 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c136cd-9335-499d-b948-918ba5726c49 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:26.793549 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c136cd-9335-499d-b948-918ba5726c49 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:26.793700 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c136cd-9335-499d-b948-918ba5726c49 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:26.796405 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 514/1032] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:55:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:26.805758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75197fa-612f-4061-bd6c-de4edca1d60c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:26.806593 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75197fa-612f-4061-bd6c-de4edca1d60c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:26.806593 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75197fa-612f-4061-bd6c-de4edca1d60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:26.841168 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-f75197fa-612f-4061-bd6c-de4edca1d60c None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:26.843207 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f75197fa-612f-4061-bd6c-de4edca1d60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:26.869910 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:26.953144 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 519/1033] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:55:26.987693 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c136cd-9335-499d-b948-918ba5726c49 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:27.186876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c136cd-9335-499d-b948-918ba5726c49 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:27.190373 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:27.257320 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22b7ca-8386-49fb-928d-1ba180d48425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:27.258368 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22b7ca-8386-49fb-928d-1ba180d48425 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:27.258777 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22b7ca-8386-49fb-928d-1ba180d48425 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:27.259178 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22b7ca-8386-49fb-928d-1ba180d48425 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:27.283691 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22b7ca-8386-49fb-928d-1ba180d48425 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:27.292624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-da22b7ca-8386-49fb-928d-1ba180d48425 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 13:55:27.292624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:27.292624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:27.292624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:27.292624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:27.292624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:27.292624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:27.292624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:27.292624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:27.292624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:27.292624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:27.292624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:27.292624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:27.292624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:27.292624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:27.292624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:27.293795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:27.293795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:27.293795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:27.293795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:27.293795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:27.293795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:27.293795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:27.293795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:27.293795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:27.293795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:27.293795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:27.293795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:27.293795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:27.293795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:27.293795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:27.293795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:27.294853 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:27.294853 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:27.294853 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:27.294853 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:27.294853 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:27.294853 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:27.294853 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:27.294853 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:27.294853 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:27.294853 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:27.294853 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 13:55:27.294853 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:27.296302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 515/1034] 10.222.0.45 () {56 vars in 1120 bytes} [Mon Aug 1 13:55:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:27.304255 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75197fa-612f-4061-bd6c-de4edca1d60c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:27.481041 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75197fa-612f-4061-bd6c-de4edca1d60c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:27.552131 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598cf87-ebb1-474e-a531-eb6c73e58399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:27.553198 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598cf87-ebb1-474e-a531-eb6c73e58399 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:27.553663 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598cf87-ebb1-474e-a531-eb6c73e58399 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:27.553978 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598cf87-ebb1-474e-a531-eb6c73e58399 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:27.554903 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598cf87-ebb1-474e-a531-eb6c73e58399 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:27.555416 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598cf87-ebb1-474e-a531-eb6c73e58399 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:27.557971 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598cf87-ebb1-474e-a531-eb6c73e58399 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:27.580738 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 520/1035] 10.222.0.45 () {56 vars in 1115 bytes} [Mon Aug 1 13:55:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:27.583782 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22b7ca-8386-49fb-928d-1ba180d48425 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:27.791040 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22b7ca-8386-49fb-928d-1ba180d48425 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:27.878778 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871600fa-b938-4149-b8b3-68a76ac81a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:27.879818 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871600fa-b938-4149-b8b3-68a76ac81a69 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:27.880233 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871600fa-b938-4149-b8b3-68a76ac81a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:27.880674 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871600fa-b938-4149-b8b3-68a76ac81a69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:27.900772 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871600fa-b938-4149-b8b3-68a76ac81a69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:27.902342 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871600fa-b938-4149-b8b3-68a76ac81a69 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:27.932926 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-871600fa-b938-4149-b8b3-68a76ac81a69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:55:27.932926 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:27.932926 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:27.932926 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:27.932926 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:27.932926 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:27.932926 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:27.932926 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:27.932926 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:27.932926 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:27.932926 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:27.932926 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:27.932926 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:55:27.932926 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:55:27.932926 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:55:27.934668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:55:27.934668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:27.934668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:27.934668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:27.934668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:27.934668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:27.934668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:27.934668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:27.934668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:27.934668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:27.934668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:27.934668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:27.934668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:27.934668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:27.934668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:27.934668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:27.934668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:27.936224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:55:27.936224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:55:27.936224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:55:27.936224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:27.940764 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 516/1036] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:27.945343 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598cf87-ebb1-474e-a531-eb6c73e58399 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:27.979054 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:28.152128 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598cf87-ebb1-474e-a531-eb6c73e58399 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:28.218364 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e00e8b-98c1-47d7-b50f-ad5e8a938a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:28.219544 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e00e8b-98c1-47d7-b50f-ad5e8a938a4b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:28.219988 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e00e8b-98c1-47d7-b50f-ad5e8a938a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:28.220527 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e00e8b-98c1-47d7-b50f-ad5e8a938a4b None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:28.221867 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e00e8b-98c1-47d7-b50f-ad5e8a938a4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:28.222411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e00e8b-98c1-47d7-b50f-ad5e8a938a4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:28.224848 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e00e8b-98c1-47d7-b50f-ad5e8a938a4b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:28.248318 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 521/1037] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:55:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:28.257998 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871600fa-b938-4149-b8b3-68a76ac81a69 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:28.288630 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:28.448171 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871600fa-b938-4149-b8b3-68a76ac81a69 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:28.492664 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f125a-c5a6-4f0c-8c1d-0f7d167f3ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:28.493533 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f125a-c5a6-4f0c-8c1d-0f7d167f3ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:28.493963 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f125a-c5a6-4f0c-8c1d-0f7d167f3ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:28.494369 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f125a-c5a6-4f0c-8c1d-0f7d167f3ab6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:28.505185 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f125a-c5a6-4f0c-8c1d-0f7d167f3ab6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:28.507139 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f125a-c5a6-4f0c-8c1d-0f7d167f3ab6 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:28.520727 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-341f125a-c5a6-4f0c-8c1d-0f7d167f3ab6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:55:28.520727 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:28.520727 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:28.520727 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:28.520727 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:28.520727 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:28.520727 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:28.520727 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:28.520727 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:28.520727 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:28.520727 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:28.520727 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:28.520727 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:55:28.520727 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:55:28.520727 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:55:28.520727 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:55:28.522992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:28.522992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:28.522992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:55:28.522992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:55:28.522992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:55:28.522992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:55:28.522992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:28.522992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:28.522992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:28.522992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:28.522992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:28.522992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:28.522992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:28.522992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:28.522992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:28.522992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:28.522992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:28.522992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:28.531572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:28.531572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:28.531572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:55:28.531572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:55:28.531572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:55:28.531572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:55:28.531572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:55:28.531572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:55:28.531572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:55:28.531572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:28.531572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 517/1038] 10.222.0.45 () {56 vars in 1093 bytes} [Mon Aug 1 13:55:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:55:28.532991 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e00e8b-98c1-47d7-b50f-ad5e8a938a4b None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:28.677129 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:28.740294 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e00e8b-98c1-47d7-b50f-ad5e8a938a4b None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:28.813973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396677d5-5ef2-4512-91ff-fa741b662c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:28.815209 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396677d5-5ef2-4512-91ff-fa741b662c7a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:28.815809 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396677d5-5ef2-4512-91ff-fa741b662c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:28.816434 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396677d5-5ef2-4512-91ff-fa741b662c7a None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:28.818085 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396677d5-5ef2-4512-91ff-fa741b662c7a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:28.818754 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396677d5-5ef2-4512-91ff-fa741b662c7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:28.821630 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396677d5-5ef2-4512-91ff-fa741b662c7a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:28.861594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 522/1039] 10.222.0.45 () {56 vars in 1133 bytes} [Mon Aug 1 13:55:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:28.863241 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f125a-c5a6-4f0c-8c1d-0f7d167f3ab6 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:28.990248 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:29.028900 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f125a-c5a6-4f0c-8c1d-0f7d167f3ab6 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:29.085007 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e7fd7-bca2-4e86-a296-93e197c10c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:29.086058 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e7fd7-bca2-4e86-a296-93e197c10c14 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:29.086524 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e7fd7-bca2-4e86-a296-93e197c10c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:29.087004 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e7fd7-bca2-4e86-a296-93e197c10c14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:29.225855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375e7fd7-bca2-4e86-a296-93e197c10c14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:29.228773 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375e7fd7-bca2-4e86-a296-93e197c10c14 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:29.232392 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc8e28-9a3b-46d6-9ae2-fb2db3de0237 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:29.232959 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc8e28-9a3b-46d6-9ae2-fb2db3de0237 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:29.233279 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc8e28-9a3b-46d6-9ae2-fb2db3de0237 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:29.235705 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 523/1040] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:55:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:55:29.245310 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb1659-9ed3-4a2a-a188-655daa6890a6 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:29.245813 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb1659-9ed3-4a2a-a188-655daa6890a6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:29.246310 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb1659-9ed3-4a2a-a188-655daa6890a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:29.277611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-63fb1659-9ed3-4a2a-a188-655daa6890a6 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:29.279154 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-63fb1659-9ed3-4a2a-a188-655daa6890a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:29.359653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:29.399694 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 518/1041] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:28 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:55:29.402431 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e7fd7-bca2-4e86-a296-93e197c10c14 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:29.404036 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 524/1042] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:55:29.426421 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb1659-9ed3-4a2a-a188-655daa6890a6 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:29.503048 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:29.566293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e7fd7-bca2-4e86-a296-93e197c10c14 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:29.581843 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb1659-9ed3-4a2a-a188-655daa6890a6 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:29.629954 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c669b01-2231-401a-a495-abecf6f4d6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:29.630786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c669b01-2231-401a-a495-abecf6f4d6f4 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:29.631077 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c669b01-2231-401a-a495-abecf6f4d6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:29.631392 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c669b01-2231-401a-a495-abecf6f4d6f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:29.641969 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf62be5-468c-4b8d-a08b-074efd290be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:29.642817 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf62be5-468c-4b8d-a08b-074efd290be3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:29.643161 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf62be5-468c-4b8d-a08b-074efd290be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:29.643432 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf62be5-468c-4b8d-a08b-074efd290be3 None admin] PATH_INFO: `/v3/services/image` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:29.644147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf62be5-468c-4b8d-a08b-074efd290be3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:29.645696 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c669b01-2231-401a-a495-abecf6f4d6f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:29.645878 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf62be5-468c-4b8d-a08b-074efd290be3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:29.647935 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c669b01-2231-401a-a495-abecf6f4d6f4 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:29.659701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-5bf62be5-468c-4b8d-a08b-074efd290be3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 13:55:29.659701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:29.659701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:55:29.659701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:55:29.659701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:55:29.659701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:55:29.659701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:55:29.659701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:55:29.659701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 13:55:29.659701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:29.659701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:55:29.659701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:29.659701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:29.659701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:29.659701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:29.659701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:29.659701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:29.660849 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:29.660849 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:29.660849 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:29.660849 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:29.660849 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:29.660849 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:29.660849 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:55:29.660849 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:55:29.660849 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:55:29.660849 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:55:29.660849 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:29.660849 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:29.660849 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:29.660849 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:29.660849 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:29.660849 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:29.661714 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:29.661714 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:29.661714 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:29.661714 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:29.661714 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:29.661714 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:29.661714 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:29.661714 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:29.661714 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:29.661714 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:29.661714 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:55:29.661714 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:55:29.661714 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 13:55:29.661714 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:29.662464 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 525/1043] 10.222.0.45 () {56 vars in 1105 bytes} [Mon Aug 1 13:55:29 2022] GET /identity/v3/services/image => generated 86 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:55:29.664182 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58725ee5-0fc7-4bd4-9da8-fd60c5141f38 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:29.664768 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58725ee5-0fc7-4bd4-9da8-fd60c5141f38 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:29.665222 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58725ee5-0fc7-4bd4-9da8-fd60c5141f38 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:29.667083 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 526/1044] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:55:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:29.668860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-6c669b01-2231-401a-a495-abecf6f4d6f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:55:29.668860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:29.668860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:29.668860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:29.668860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:29.668860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:29.668860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:29.668860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:29.668860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:29.668860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:29.668860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:29.668860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:29.668860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:55:29.668860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:55:29.668860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:55:29.670767 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:55:29.670767 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:29.670767 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:29.670767 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:29.670767 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:29.670767 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:29.670767 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:29.670767 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:29.670767 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:29.670767 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:29.670767 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:29.670767 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:29.670767 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:29.670767 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:29.670767 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:29.670767 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:29.670767 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:29.672387 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:55:29.672387 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:55:29.672387 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:55:29.672387 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:29.672387 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 519/1045] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:29.676817 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09a566-a94e-449e-a12a-76573a55a98a None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:29.676988 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58725ee5-0fc7-4bd4-9da8-fd60c5141f38 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:29.677114 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09a566-a94e-449e-a12a-76573a55a98a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:29.677114 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09a566-a94e-449e-a12a-76573a55a98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:29.694898 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-8f09a566-a94e-449e-a12a-76573a55a98a None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:29.696150 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-8f09a566-a94e-449e-a12a-76573a55a98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:29.725353 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:29.813322 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 520/1046] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:55:29.814807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09a566-a94e-449e-a12a-76573a55a98a None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:29.855793 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58725ee5-0fc7-4bd4-9da8-fd60c5141f38 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:29.904554 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:29.926507 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04307918-4ccf-44ee-a8b7-2e274578deba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:29.927482 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04307918-4ccf-44ee-a8b7-2e274578deba None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:29.927915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04307918-4ccf-44ee-a8b7-2e274578deba None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:29.928390 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04307918-4ccf-44ee-a8b7-2e274578deba None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:29.929520 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04307918-4ccf-44ee-a8b7-2e274578deba None admin] RBAC: Adding query filter params (name=image) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:29.930018 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04307918-4ccf-44ee-a8b7-2e274578deba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:29.932456 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04307918-4ccf-44ee-a8b7-2e274578deba None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:29.941580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 527/1047] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:29.944980 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04307918-4ccf-44ee-a8b7-2e274578deba None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:30.016165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09a566-a94e-449e-a12a-76573a55a98a None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:30.076205 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b928cd2-54fb-4207-8ec7-9c13da265c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:30.077871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b928cd2-54fb-4207-8ec7-9c13da265c37 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:30.078474 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b928cd2-54fb-4207-8ec7-9c13da265c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:30.079039 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b928cd2-54fb-4207-8ec7-9c13da265c37 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:30.080939 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b928cd2-54fb-4207-8ec7-9c13da265c37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:30.081425 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b928cd2-54fb-4207-8ec7-9c13da265c37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:30.083613 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b928cd2-54fb-4207-8ec7-9c13da265c37 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:30.108869 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 521/1048] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:30.108869 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b928cd2-54fb-4207-8ec7-9c13da265c37 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:30.132999 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:30.139067 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04307918-4ccf-44ee-a8b7-2e274578deba None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:30.200898 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89de2bf-70b3-4590-a3ae-88e841148879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:30.202056 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89de2bf-70b3-4590-a3ae-88e841148879 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:30.202531 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89de2bf-70b3-4590-a3ae-88e841148879 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:30.203054 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89de2bf-70b3-4590-a3ae-88e841148879 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:30.222879 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89de2bf-70b3-4590-a3ae-88e841148879 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:30.232911 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-e89de2bf-70b3-4590-a3ae-88e841148879 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:55:30.232911 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:30.232911 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:30.232911 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:30.232911 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:30.232911 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:30.232911 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:30.232911 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:30.232911 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:30.232911 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:30.232911 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:30.232911 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:30.232911 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:30.232911 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:30.232911 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:30.232911 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:30.235128 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:30.235128 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:30.235128 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:30.235128 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:30.235128 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:30.235128 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:30.235128 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:30.235128 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:30.235128 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:30.235128 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:30.235128 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:30.235128 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:30.235128 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:30.235128 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:30.235128 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:30.235128 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:30.236054 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:30.236054 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:30.236054 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:30.236054 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:30.236054 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:30.236054 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:30.236054 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:30.236054 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:30.236054 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:30.236054 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:30.236054 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:55:30.236054 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:30.236054 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 528/1049] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:55:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:30.238059 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89de2bf-70b3-4590-a3ae-88e841148879 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:30.262448 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b928cd2-54fb-4207-8ec7-9c13da265c37 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:30.321369 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b438b-c5f0-4479-8a86-1c1b4ce7cfc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:30.322378 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b438b-c5f0-4479-8a86-1c1b4ce7cfc7 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:30.322802 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b438b-c5f0-4479-8a86-1c1b4ce7cfc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:30.323257 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b438b-c5f0-4479-8a86-1c1b4ce7cfc7 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:30.324545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b438b-c5f0-4479-8a86-1c1b4ce7cfc7 None admin] RBAC: Adding query filter params (type=image) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:30.325263 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b438b-c5f0-4479-8a86-1c1b4ce7cfc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:30.327966 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b438b-c5f0-4479-8a86-1c1b4ce7cfc7 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:30.329827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:30.337156 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 522/1050] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:30.338409 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b438b-c5f0-4479-8a86-1c1b4ce7cfc7 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:30.413396 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89de2bf-70b3-4590-a3ae-88e841148879 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:30.457522 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3832a8f-9a66-4ad7-9f77-745ea90c7b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:30.458208 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3832a8f-9a66-4ad7-9f77-745ea90c7b21 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:30.458475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3832a8f-9a66-4ad7-9f77-745ea90c7b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:30.458770 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3832a8f-9a66-4ad7-9f77-745ea90c7b21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:30.467515 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3832a8f-9a66-4ad7-9f77-745ea90c7b21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:30.469309 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:30.469516 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3832a8f-9a66-4ad7-9f77-745ea90c7b21 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:30.482167 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-a3832a8f-9a66-4ad7-9f77-745ea90c7b21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:55:30.482167 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:30.482167 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:30.482167 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:30.482167 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:30.482167 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:30.482167 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:30.482167 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:30.482167 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:30.482167 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:30.482167 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:30.482167 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:30.482167 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:55:30.482167 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:55:30.482167 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:55:30.484095 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:55:30.484095 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:30.484095 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:30.484095 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:30.484095 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:30.484095 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:30.484095 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:30.484095 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:30.484095 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:30.484095 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:30.484095 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:30.484095 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:30.484095 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:30.484095 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:30.484095 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:30.484095 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:30.484095 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:30.485585 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:55:30.485585 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:55:30.485585 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:55:30.485585 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:55:30.485585 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:55:30.485585 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:55:30.485585 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:55:30.485585 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:30.488585 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 529/1051] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:55:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:30.490235 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3832a8f-9a66-4ad7-9f77-745ea90c7b21 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:30.501058 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b438b-c5f0-4479-8a86-1c1b4ce7cfc7 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:30.560937 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d9068-b7dc-4716-b3b9-f40e64471082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:30.561603 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d9068-b7dc-4716-b3b9-f40e64471082 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:30.561915 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d9068-b7dc-4716-b3b9-f40e64471082 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:30.562245 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d9068-b7dc-4716-b3b9-f40e64471082 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:30.563071 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6d9068-b7dc-4716-b3b9-f40e64471082 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:30.563399 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6d9068-b7dc-4716-b3b9-f40e64471082 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:30.565302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6d9068-b7dc-4716-b3b9-f40e64471082 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:30.576625 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 523/1052] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:30.577501 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d9068-b7dc-4716-b3b9-f40e64471082 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:30.642850 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3832a8f-9a66-4ad7-9f77-745ea90c7b21 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:30.659309 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:30.704621 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7fd4ae-7049-44cb-b251-1bbc921d1634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:30.705584 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fd4ae-7049-44cb-b251-1bbc921d1634 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:30.706108 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fd4ae-7049-44cb-b251-1bbc921d1634 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:30.706543 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fd4ae-7049-44cb-b251-1bbc921d1634 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:30.707727 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7fd4ae-7049-44cb-b251-1bbc921d1634 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:30.710937 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7fd4ae-7049-44cb-b251-1bbc921d1634 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:30.743644 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 530/1053] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 13:55:30 2022] POST /identity/v3/endpoints => generated 327 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:55:30.745581 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7fd4ae-7049-44cb-b251-1bbc921d1634 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:30.764201 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d9068-b7dc-4716-b3b9-f40e64471082 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:30.845413 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14054291-9ab2-4181-a839-e2e80b0af621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:30.846787 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14054291-9ab2-4181-a839-e2e80b0af621 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:30.847200 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14054291-9ab2-4181-a839-e2e80b0af621 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:30.847677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14054291-9ab2-4181-a839-e2e80b0af621 None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:30.849197 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14054291-9ab2-4181-a839-e2e80b0af621 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:30.849659 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14054291-9ab2-4181-a839-e2e80b0af621 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:30.852228 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14054291-9ab2-4181-a839-e2e80b0af621 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:30.866022 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:30.882313 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 524/1054] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:55:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:30.887093 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14054291-9ab2-4181-a839-e2e80b0af621 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:30.992573 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7fd4ae-7049-44cb-b251-1bbc921d1634 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:31.061326 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22204cac-421c-4c49-881d-fd6e4b44e794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:31.062027 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22204cac-421c-4c49-881d-fd6e4b44e794 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:31.062390 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22204cac-421c-4c49-881d-fd6e4b44e794 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:31.062688 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22204cac-421c-4c49-881d-fd6e4b44e794 None admin] PATH_INFO: `/v3/users/e8d61e11850b4d8f855a2926ed6c79ce` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:31.083669 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22204cac-421c-4c49-881d-fd6e4b44e794 None admin] RBAC: Authorizing `identity:get_user(user_id=e8d61e11850b4d8f855a2926ed6c79ce)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:31.085807 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22204cac-421c-4c49-881d-fd6e4b44e794 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:31.088730 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 531/1055] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:30 2022] GET /identity/v3/users/e8d61e11850b4d8f855a2926ed6c79ce => generated 285 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:31.096438 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22204cac-421c-4c49-881d-fd6e4b44e794 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:31.120362 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14054291-9ab2-4181-a839-e2e80b0af621 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:31.148300 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:31.187852 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b751fc8-4eed-4d58-baf7-66b48033d2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:31.188956 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b751fc8-4eed-4d58-baf7-66b48033d2bb None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:31.189450 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b751fc8-4eed-4d58-baf7-66b48033d2bb None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:31.189926 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b751fc8-4eed-4d58-baf7-66b48033d2bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:31.191826 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b751fc8-4eed-4d58-baf7-66b48033d2bb None admin] RBAC: Adding query filter params (role.id=c29b2bae1d6c4e438cb861ed195d5c72, scope.project.id=36f063d8f65d494a89dbbd72c53417c8, user.id=5b3adc14dcaf4ac79b43bb97fa1ba1d4) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:31.192405 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b751fc8-4eed-4d58-baf7-66b48033d2bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:31.195048 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b751fc8-4eed-4d58-baf7-66b48033d2bb None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:31.205670 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 525/1056] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:55:30 2022] GET /identity/v3/role_assignments?user.id=5b3adc14dcaf4ac79b43bb97fa1ba1d4&scope.project.id=36f063d8f65d494a89dbbd72c53417c8&role.id=c29b2bae1d6c4e438cb861ed195d5c72 => generated 259 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:31.323698 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22204cac-421c-4c49-881d-fd6e4b44e794 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:31.397421 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bbb510-49e0-45d6-bbfd-405c23d9ced6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:31.398344 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbb510-49e0-45d6-bbfd-405c23d9ced6 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:31.398754 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbb510-49e0-45d6-bbfd-405c23d9ced6 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:31.399200 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbb510-49e0-45d6-bbfd-405c23d9ced6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:31.548718 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bbb510-49e0-45d6-bbfd-405c23d9ced6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:31.551476 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bbb510-49e0-45d6-bbfd-405c23d9ced6 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:31.601581 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:31.746481 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 532/1057] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 651 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 13:55:31.755676 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b751fc8-4eed-4d58-baf7-66b48033d2bb None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:31.878144 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:31.967536 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b751fc8-4eed-4d58-baf7-66b48033d2bb None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:32.063339 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa4c2f6-89b8-458e-b881-c79abb1a3df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:32.064587 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa4c2f6-89b8-458e-b881-c79abb1a3df6 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:32.065072 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa4c2f6-89b8-458e-b881-c79abb1a3df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:32.065530 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa4c2f6-89b8-458e-b881-c79abb1a3df6 None admin] PATH_INFO: `/v3/projects/e9050cf3677442a6a5d5bf54ad0dc036` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:32.067798 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa4c2f6-89b8-458e-b881-c79abb1a3df6 None admin] RBAC: Authorizing `identity:get_project(project_id=e9050cf3677442a6a5d5bf54ad0dc036)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:32.070585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa4c2f6-89b8-458e-b881-c79abb1a3df6 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:32.073867 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 526/1058] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:31 2022] GET /identity/v3/projects/e9050cf3677442a6a5d5bf54ad0dc036 => generated 311 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:32.084602 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bbb510-49e0-45d6-bbfd-405c23d9ced6 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:32.313222 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bbb510-49e0-45d6-bbfd-405c23d9ced6 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:32.355620 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:32.380174 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1803cd77-c72d-4ee3-bf8c-971a82f79315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:32.381290 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803cd77-c72d-4ee3-bf8c-971a82f79315 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:32.381765 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803cd77-c72d-4ee3-bf8c-971a82f79315 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:32.382196 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803cd77-c72d-4ee3-bf8c-971a82f79315 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:32.383580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1803cd77-c72d-4ee3-bf8c-971a82f79315 None admin] RBAC: Adding query filter params (role.id=3e1a9ff016c840c5b51ef7e3da9900ac, scope.project.id=e9050cf3677442a6a5d5bf54ad0dc036, user.id=e8d61e11850b4d8f855a2926ed6c79ce) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:32.384096 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1803cd77-c72d-4ee3-bf8c-971a82f79315 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:32.386830 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1803cd77-c72d-4ee3-bf8c-971a82f79315 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:32.404313 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 533/1059] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:55:32 2022] GET /identity/v3/role_assignments?user.id=e8d61e11850b4d8f855a2926ed6c79ce&scope.project.id=e9050cf3677442a6a5d5bf54ad0dc036&role.id=3e1a9ff016c840c5b51ef7e3da9900ac => generated 259 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:34.111915 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc49dc1-3c5e-4137-8283-d62307f90545 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:34.112869 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc49dc1-3c5e-4137-8283-d62307f90545 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:34.112869 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc49dc1-3c5e-4137-8283-d62307f90545 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:34.115328 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 527/1060] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:55:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:34.123968 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c75d0b-238c-448b-bac5-0eda6acc390c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:34.124862 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c75d0b-238c-448b-bac5-0eda6acc390c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:34.125181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c75d0b-238c-448b-bac5-0eda6acc390c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:34.157389 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-44c75d0b-238c-448b-bac5-0eda6acc390c None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:34.159446 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-44c75d0b-238c-448b-bac5-0eda6acc390c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:34.183402 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8033d4-d430-4bc4-af01-80a98dbc4449 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:34.183678 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8033d4-d430-4bc4-af01-80a98dbc4449 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:34.184078 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8033d4-d430-4bc4-af01-80a98dbc4449 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:34.185716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 528/1061] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:55:34.191703 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b807b4-2118-4fa8-96e9-9960fbbaf0f2 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:34.191967 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b807b4-2118-4fa8-96e9-9960fbbaf0f2 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:34.192250 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b807b4-2118-4fa8-96e9-9960fbbaf0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:34.216032 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-c6b807b4-2118-4fa8-96e9-9960fbbaf0f2 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:34.218102 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c6b807b4-2118-4fa8-96e9-9960fbbaf0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:34.302213 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 534/1062] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:55:34.327026 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c75d0b-238c-448b-bac5-0eda6acc390c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:34.337630 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 529/1063] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:55:34.381950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b807b4-2118-4fa8-96e9-9960fbbaf0f2 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:34.493387 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:34.514926 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c75d0b-238c-448b-bac5-0eda6acc390c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:34.552497 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:34.562155 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b807b4-2118-4fa8-96e9-9960fbbaf0f2 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:34.590171 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb455d9-80ed-43d0-bc4f-3a69218c52b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:34.591287 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb455d9-80ed-43d0-bc4f-3a69218c52b0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:34.591762 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb455d9-80ed-43d0-bc4f-3a69218c52b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:34.592248 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb455d9-80ed-43d0-bc4f-3a69218c52b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:34.637746 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a999d-aecc-4723-ade0-57e97110e2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:34.638842 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a999d-aecc-4723-ade0-57e97110e2f9 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:34.639203 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a999d-aecc-4723-ade0-57e97110e2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:34.639638 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a999d-aecc-4723-ade0-57e97110e2f9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:34.663250 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3a999d-aecc-4723-ade0-57e97110e2f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:34.674871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-0d3a999d-aecc-4723-ade0-57e97110e2f9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 13:55:34.674871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:34.674871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:34.674871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:34.674871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:34.674871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:34.674871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:34.674871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:34.674871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:34.674871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:34.674871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:34.674871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:34.674871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:34.674871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:34.674871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:34.674871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:34.677121 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:34.677121 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:34.677121 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:34.677121 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:34.677121 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:34.677121 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:34.677121 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:34.677121 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:34.677121 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:34.677121 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:34.677121 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:34.677121 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:34.677121 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:34.677121 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:34.677121 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:34.677121 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:34.679125 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:34.679125 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:34.679125 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:34.679125 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:34.679125 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:34.679125 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:34.679125 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:34.679125 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:34.679125 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:34.679125 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:34.679125 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 13:55:34.679125 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:34.679125 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 530/1064] 10.222.0.45 () {56 vars in 1120 bytes} [Mon Aug 1 13:55:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:34.683112 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a999d-aecc-4723-ade0-57e97110e2f9 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:34.739153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb455d9-80ed-43d0-bc4f-3a69218c52b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:34.741839 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb455d9-80ed-43d0-bc4f-3a69218c52b0 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:34.872878 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a999d-aecc-4723-ade0-57e97110e2f9 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:34.918868 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:34.948893 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd3b6d-3e8e-4b97-83d7-45763d17fb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:34.950026 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd3b6d-3e8e-4b97-83d7-45763d17fb81 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:34.955724 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd3b6d-3e8e-4b97-83d7-45763d17fb81 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:34.956026 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 535/1065] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 13:55:34.956177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd3b6d-3e8e-4b97-83d7-45763d17fb81 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:34.956177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dd3b6d-3e8e-4b97-83d7-45763d17fb81 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:34.956177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dd3b6d-3e8e-4b97-83d7-45763d17fb81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:34.956177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dd3b6d-3e8e-4b97-83d7-45763d17fb81 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:34.957447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb455d9-80ed-43d0-bc4f-3a69218c52b0 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:34.971865 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 531/1066] 10.222.0.45 () {56 vars in 1115 bytes} [Mon Aug 1 13:55:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:34.981558 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd3b6d-3e8e-4b97-83d7-45763d17fb81 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:34.989325 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:35.139312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd3b6d-3e8e-4b97-83d7-45763d17fb81 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:35.151793 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb455d9-80ed-43d0-bc4f-3a69218c52b0 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:35.191213 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b9163-f09b-4f19-990f-d5c4d6ed23d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:35.191790 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b9163-f09b-4f19-990f-d5c4d6ed23d2 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:35.192071 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b9163-f09b-4f19-990f-d5c4d6ed23d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:35.192380 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b9163-f09b-4f19-990f-d5c4d6ed23d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:35.207599 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b9163-f09b-4f19-990f-d5c4d6ed23d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:35.211559 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2174d10-db5f-4396-9917-2d1acc6cce18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:35.212549 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2174d10-db5f-4396-9917-2d1acc6cce18 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:35.212880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b9163-f09b-4f19-990f-d5c4d6ed23d2 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:35.213039 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2174d10-db5f-4396-9917-2d1acc6cce18 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:35.213369 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2174d10-db5f-4396-9917-2d1acc6cce18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:35.228981 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2174d10-db5f-4396-9917-2d1acc6cce18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:35.231651 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2174d10-db5f-4396-9917-2d1acc6cce18 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:35.243746 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-745b9163-f09b-4f19-990f-d5c4d6ed23d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:55:35.243746 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:35.243746 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:35.243746 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:35.243746 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:35.243746 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:35.243746 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:35.243746 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:35.243746 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:35.243746 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:35.243746 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:35.243746 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:35.243746 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:55:35.243746 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:55:35.243746 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:55:35.245682 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:55:35.245682 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:35.245682 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:35.245682 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:35.245682 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:35.245682 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:35.245682 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:35.245682 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:35.245682 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:35.245682 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:35.245682 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:35.245682 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:35.245682 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:35.245682 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:35.245682 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:35.245682 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:35.245682 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:35.247382 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:55:35.247382 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:55:35.247382 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:55:35.247382 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:35.247382 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 532/1067] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:55:35.254683 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b9163-f09b-4f19-990f-d5c4d6ed23d2 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:35.265501 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-b2174d10-db5f-4396-9917-2d1acc6cce18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:55:35.265501 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:35.265501 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:35.265501 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:35.265501 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:35.265501 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:35.265501 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:35.265501 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:35.265501 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:35.265501 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:35.265501 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:35.265501 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:35.265501 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:55:35.265501 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:55:35.265501 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:55:35.277492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:55:35.277492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:35.277492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:35.277492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:35.277492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:35.277492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:35.277492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:35.277492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:35.277492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:35.277492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:35.277492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:35.277492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:35.277492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:35.277492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:35.277492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:35.277492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:35.277492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:35.279262 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:55:35.279262 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:55:35.279262 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:55:35.279262 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:35.279262 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 536/1068] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:35.279262 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2174d10-db5f-4396-9917-2d1acc6cce18 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:35.319305 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:35.391095 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:35.431835 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2174d10-db5f-4396-9917-2d1acc6cce18 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:35.436108 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b9163-f09b-4f19-990f-d5c4d6ed23d2 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:35.480828 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe795ad-fc3c-4af7-b588-e7d98a82e35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:35.481466 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe795ad-fc3c-4af7-b588-e7d98a82e35a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:35.481612 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921c9ca-32b5-45dc-b7a9-67da14da759d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:35.481764 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe795ad-fc3c-4af7-b588-e7d98a82e35a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:35.482012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe795ad-fc3c-4af7-b588-e7d98a82e35a None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:35.482626 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921c9ca-32b5-45dc-b7a9-67da14da759d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:35.482777 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe795ad-fc3c-4af7-b588-e7d98a82e35a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:35.482998 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe795ad-fc3c-4af7-b588-e7d98a82e35a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:35.483144 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921c9ca-32b5-45dc-b7a9-67da14da759d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:35.483583 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921c9ca-32b5-45dc-b7a9-67da14da759d None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:35.484446 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe795ad-fc3c-4af7-b588-e7d98a82e35a None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:35.484898 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7921c9ca-32b5-45dc-b7a9-67da14da759d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:35.485452 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7921c9ca-32b5-45dc-b7a9-67da14da759d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:35.487943 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7921c9ca-32b5-45dc-b7a9-67da14da759d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:35.505876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 533/1069] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:55:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:35.505876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77614c91-f825-47ad-bc0c-928bbbcdbe29 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:35.505876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77614c91-f825-47ad-bc0c-928bbbcdbe29 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:35.505876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77614c91-f825-47ad-bc0c-928bbbcdbe29 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:35.505876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 534/1070] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:55:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:35.507590 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77614c91-f825-47ad-bc0c-928bbbcdbe29 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:35.516189 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 537/1071] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:55:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:35.517898 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d72174-6323-4f82-82b3-636594eaf260 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:35.518197 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d72174-6323-4f82-82b3-636594eaf260 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:35.518478 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d72174-6323-4f82-82b3-636594eaf260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:35.544593 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-82d72174-6323-4f82-82b3-636594eaf260 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:35.546196 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-82d72174-6323-4f82-82b3-636594eaf260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:35.643106 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 538/1072] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:55:35.684382 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d72174-6323-4f82-82b3-636594eaf260 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:35.716591 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:35.734590 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77614c91-f825-47ad-bc0c-928bbbcdbe29 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:35.795993 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74589e0-7e68-40ca-ace2-b43cae3c5052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:35.795993 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74589e0-7e68-40ca-ace2-b43cae3c5052 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:35.796366 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74589e0-7e68-40ca-ace2-b43cae3c5052 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:35.796626 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74589e0-7e68-40ca-ace2-b43cae3c5052 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:35.802162 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:35.806928 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74589e0-7e68-40ca-ace2-b43cae3c5052 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:35.808460 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74589e0-7e68-40ca-ace2-b43cae3c5052 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:35.826972 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-d74589e0-7e68-40ca-ace2-b43cae3c5052 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:55:35.826972 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:35.826972 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:35.826972 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:35.826972 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:35.826972 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:35.826972 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:35.826972 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:35.826972 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:35.826972 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:35.826972 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:35.826972 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:35.826972 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:55:35.826972 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:55:35.826972 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:55:35.826972 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:55:35.829062 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:35.829062 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:35.829062 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:55:35.829062 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:55:35.829062 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:55:35.829062 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:55:35.829062 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:35.829062 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:35.829062 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:35.829062 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:35.829062 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:35.829062 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:35.829062 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:35.829062 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:35.829062 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:35.829062 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:35.829062 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:35.829062 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:35.833919 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:35.833919 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:35.833919 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:55:35.833919 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:55:35.833919 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:55:35.833919 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:55:35.833919 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:55:35.833919 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:55:35.833919 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:55:35.833919 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:35.837243 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 535/1073] 10.222.0.45 () {56 vars in 1093 bytes} [Mon Aug 1 13:55:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:35.838445 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74589e0-7e68-40ca-ace2-b43cae3c5052 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:35.902640 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d72174-6323-4f82-82b3-636594eaf260 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:35.948227 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b5882a-ca6b-4da5-90e0-ef6d87db5a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:35.949646 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5882a-ca6b-4da5-90e0-ef6d87db5a4d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:35.950075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5882a-ca6b-4da5-90e0-ef6d87db5a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:35.950472 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5882a-ca6b-4da5-90e0-ef6d87db5a4d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:35.965171 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b5882a-ca6b-4da5-90e0-ef6d87db5a4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:35.971858 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-c9b5882a-ca6b-4da5-90e0-ef6d87db5a4d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:55:35.971858 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:35.971858 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:35.971858 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:35.971858 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:35.971858 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:35.971858 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:35.971858 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:35.971858 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:35.971858 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:35.971858 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:35.971858 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:35.971858 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:35.971858 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:35.971858 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:35.971858 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:35.973125 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:35.973125 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:35.973125 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:35.973125 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:35.973125 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:35.973125 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:35.973125 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:35.973125 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:35.973125 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:35.973125 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:35.973125 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:35.973125 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:35.973125 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:35.973125 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:35.973125 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:35.973125 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:35.974188 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:35.974188 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:35.974188 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:35.974188 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:35.974188 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:35.974188 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:35.974188 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:35.974188 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:35.974188 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:35.974188 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:35.974188 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:55:35.974188 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:35.974188 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 539/1074] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:55:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:35.980293 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b5882a-ca6b-4da5-90e0-ef6d87db5a4d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:36.000176 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74589e0-7e68-40ca-ace2-b43cae3c5052 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:36.040059 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a2cafb-3a33-4dc4-a388-67cb0d413b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:36.040849 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2cafb-3a33-4dc4-a388-67cb0d413b02 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:36.041206 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2cafb-3a33-4dc4-a388-67cb0d413b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:36.041575 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2cafb-3a33-4dc4-a388-67cb0d413b02 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:36.042835 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a2cafb-3a33-4dc4-a388-67cb0d413b02 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:36.043230 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a2cafb-3a33-4dc4-a388-67cb0d413b02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:36.044913 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a2cafb-3a33-4dc4-a388-67cb0d413b02 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:36.067239 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 536/1075] 10.222.0.45 () {56 vars in 1133 bytes} [Mon Aug 1 13:55:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:36.073647 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a2cafb-3a33-4dc4-a388-67cb0d413b02 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:36.097032 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:36.126046 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b5882a-ca6b-4da5-90e0-ef6d87db5a4d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:36.147619 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:36.169766 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b266f66-4256-4bed-a091-34943c8506ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:36.170798 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b266f66-4256-4bed-a091-34943c8506ad None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:36.171139 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b266f66-4256-4bed-a091-34943c8506ad None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:36.171531 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b266f66-4256-4bed-a091-34943c8506ad None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:36.172646 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b266f66-4256-4bed-a091-34943c8506ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:36.173006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b266f66-4256-4bed-a091-34943c8506ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:36.174501 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b266f66-4256-4bed-a091-34943c8506ad None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:36.191656 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 540/1076] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:36.193039 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b266f66-4256-4bed-a091-34943c8506ad None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:36.207306 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a2cafb-3a33-4dc4-a388-67cb0d413b02 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:36.247575 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70436e6-fd59-465a-b0ea-16b49de34c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:36.248244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70436e6-fd59-465a-b0ea-16b49de34c87 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:36.248666 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70436e6-fd59-465a-b0ea-16b49de34c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:36.249139 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70436e6-fd59-465a-b0ea-16b49de34c87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:36.273462 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70436e6-fd59-465a-b0ea-16b49de34c87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:36.275203 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70436e6-fd59-465a-b0ea-16b49de34c87 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:36.298241 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-c70436e6-fd59-465a-b0ea-16b49de34c87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:55:36.298241 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:36.298241 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:36.298241 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:36.298241 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:36.298241 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:36.298241 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:36.298241 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:36.298241 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:36.298241 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:36.298241 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:36.298241 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:36.298241 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:55:36.298241 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:55:36.298241 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:55:36.299726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:55:36.299726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:36.299726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:36.299726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:36.299726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:36.299726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:36.299726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:36.299726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:36.299726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:36.299726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:36.299726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:36.299726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:36.299726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:36.299726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:36.299726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:36.299726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:36.299726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:36.301255 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:55:36.301255 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:55:36.301255 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:55:36.301255 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:36.301255 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 537/1077] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:55:36.308127 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70436e6-fd59-465a-b0ea-16b49de34c87 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:36.320478 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b266f66-4256-4bed-a091-34943c8506ad None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:36.372601 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db225ac0-a3d1-4d23-86e3-83c2f4f45062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:36.373300 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db225ac0-a3d1-4d23-86e3-83c2f4f45062 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:36.373590 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db225ac0-a3d1-4d23-86e3-83c2f4f45062 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:36.373992 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db225ac0-a3d1-4d23-86e3-83c2f4f45062 None admin] PATH_INFO: `/v3/users/e8d61e11850b4d8f855a2926ed6c79ce` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:36.376103 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db225ac0-a3d1-4d23-86e3-83c2f4f45062 None admin] RBAC: Authorizing `identity:get_user(user_id=e8d61e11850b4d8f855a2926ed6c79ce)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:36.378477 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db225ac0-a3d1-4d23-86e3-83c2f4f45062 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:36.388219 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 541/1078] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:36 2022] GET /identity/v3/users/e8d61e11850b4d8f855a2926ed6c79ce => generated 285 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:36.389747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db225ac0-a3d1-4d23-86e3-83c2f4f45062 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:36.394172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:36.442294 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70436e6-fd59-465a-b0ea-16b49de34c87 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:36.469559 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:36.485814 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a1bdd-3ea9-4b0d-b21d-3c3c27c592ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:36.486498 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a1bdd-3ea9-4b0d-b21d-3c3c27c592ea None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:36.486777 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a1bdd-3ea9-4b0d-b21d-3c3c27c592ea None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:36.487049 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a1bdd-3ea9-4b0d-b21d-3c3c27c592ea None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:36.487745 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a1bdd-3ea9-4b0d-b21d-3c3c27c592ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:36.488077 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a1bdd-3ea9-4b0d-b21d-3c3c27c592ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:36.489567 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a1bdd-3ea9-4b0d-b21d-3c3c27c592ea None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:36.505962 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 538/1079] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:55:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:36.512607 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a1bdd-3ea9-4b0d-b21d-3c3c27c592ea None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:36.553541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db225ac0-a3d1-4d23-86e3-83c2f4f45062 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:36.598147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbee732-b903-43bf-977c-104b57512b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:36.598814 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbee732-b903-43bf-977c-104b57512b79 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:36.599088 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbee732-b903-43bf-977c-104b57512b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:36.599357 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbee732-b903-43bf-977c-104b57512b79 None admin] PATH_INFO: `/v3/projects/e9050cf3677442a6a5d5bf54ad0dc036` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:36.600676 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbee732-b903-43bf-977c-104b57512b79 None admin] RBAC: Authorizing `identity:get_project(project_id=e9050cf3677442a6a5d5bf54ad0dc036)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:36.602275 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbee732-b903-43bf-977c-104b57512b79 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:36.604333 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 542/1080] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:36 2022] GET /identity/v3/projects/e9050cf3677442a6a5d5bf54ad0dc036 => generated 311 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:36.612695 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbee732-b903-43bf-977c-104b57512b79 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:36.664159 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a1bdd-3ea9-4b0d-b21d-3c3c27c592ea None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:36.695360 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:36.708852 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87a969-6046-472a-8a1e-0559029f64e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:36.709540 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87a969-6046-472a-8a1e-0559029f64e9 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:36.709816 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87a969-6046-472a-8a1e-0559029f64e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:36.710344 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87a969-6046-472a-8a1e-0559029f64e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:36.718700 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe87a969-6046-472a-8a1e-0559029f64e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:36.720744 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe87a969-6046-472a-8a1e-0559029f64e9 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:36.734271 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-fe87a969-6046-472a-8a1e-0559029f64e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:55:36.734271 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:36.734271 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:36.734271 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:36.734271 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:36.734271 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:36.734271 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:36.734271 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:36.734271 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:36.734271 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:36.734271 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:36.734271 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:36.734271 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:55:36.734271 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:55:36.734271 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:55:36.735913 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:55:36.735913 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:36.735913 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:36.735913 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:36.735913 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:36.735913 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:36.735913 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:36.735913 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:36.735913 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:36.735913 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:36.735913 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:36.735913 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:36.735913 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:36.735913 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:36.735913 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:36.735913 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:36.735913 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:36.737082 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:55:36.737082 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:55:36.737082 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:55:36.737082 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:55:36.737082 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:55:36.737082 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:55:36.737082 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:55:36.737082 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:36.737082 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 539/1081] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:55:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:55:36.742952 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87a969-6046-472a-8a1e-0559029f64e9 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:36.766314 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbee732-b903-43bf-977c-104b57512b79 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:36.795026 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:36.820793 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2037e05-896a-483a-8627-8f32d5fcdc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:36.821538 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2037e05-896a-483a-8627-8f32d5fcdc54 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:36.821904 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2037e05-896a-483a-8627-8f32d5fcdc54 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:36.822359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2037e05-896a-483a-8627-8f32d5fcdc54 None admin] PATH_INFO: `/v3/projects/e9050cf3677442a6a5d5bf54ad0dc036/users/e8d61e11850b4d8f855a2926ed6c79ce/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:36.825052 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2037e05-896a-483a-8627-8f32d5fcdc54 None admin] RBAC: Authorizing `identity:create_grant(project_id=e9050cf3677442a6a5d5bf54ad0dc036, user_id=e8d61e11850b4d8f855a2926ed6c79ce, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:36.826823 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2037e05-896a-483a-8627-8f32d5fcdc54 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:36.846493 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 543/1082] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 13:55:36 2022] PUT /identity/v3/projects/e9050cf3677442a6a5d5bf54ad0dc036/users/e8d61e11850b4d8f855a2926ed6c79ce/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 13:55:36.934035 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87a969-6046-472a-8a1e-0559029f64e9 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:36.990684 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d3d223-4101-4596-bf70-63f5d20a57a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:36.991777 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3d223-4101-4596-bf70-63f5d20a57a6 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:36.992308 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3d223-4101-4596-bf70-63f5d20a57a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:36.992685 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3d223-4101-4596-bf70-63f5d20a57a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:36.994042 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d3d223-4101-4596-bf70-63f5d20a57a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:36.994637 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d3d223-4101-4596-bf70-63f5d20a57a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:36.997434 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d3d223-4101-4596-bf70-63f5d20a57a6 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:37.021653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 540/1083] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:55:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:37.031185 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2037e05-896a-483a-8627-8f32d5fcdc54 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:37.234949 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2037e05-896a-483a-8627-8f32d5fcdc54 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:37.302014 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a43ba-f91e-4872-be88-6150507ba52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:37.302801 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a43ba-f91e-4872-be88-6150507ba52a None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:37.303209 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a43ba-f91e-4872-be88-6150507ba52a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:37.303572 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a43ba-f91e-4872-be88-6150507ba52a None admin] PATH_INFO: `/v3/projects/36f063d8f65d494a89dbbd72c53417c8/users/5b3adc14dcaf4ac79b43bb97fa1ba1d4/roles/c29b2bae1d6c4e438cb861ed195d5c72` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:37.306584 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898a43ba-f91e-4872-be88-6150507ba52a None admin] RBAC: Authorizing `identity:create_grant(project_id=36f063d8f65d494a89dbbd72c53417c8, user_id=5b3adc14dcaf4ac79b43bb97fa1ba1d4, role_id=c29b2bae1d6c4e438cb861ed195d5c72)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:37.308198 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898a43ba-f91e-4872-be88-6150507ba52a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:37.332324 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 544/1084] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 13:55:37 2022] PUT /identity/v3/projects/36f063d8f65d494a89dbbd72c53417c8/users/5b3adc14dcaf4ac79b43bb97fa1ba1d4/roles/c29b2bae1d6c4e438cb861ed195d5c72 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:55:39.340259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ef105-1c8d-48ee-a031-f6095ca7f933 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:39.341151 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ef105-1c8d-48ee-a031-f6095ca7f933 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:39.341151 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ef105-1c8d-48ee-a031-f6095ca7f933 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:39.343101 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 541/1085] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:55:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:39.352072 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98ef27-2690-4f5a-a671-027ec2e01db1 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:39.352904 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98ef27-2690-4f5a-a671-027ec2e01db1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:39.353058 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98ef27-2690-4f5a-a671-027ec2e01db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:39.384644 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8d98ef27-2690-4f5a-a671-027ec2e01db1 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:39.386713 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8d98ef27-2690-4f5a-a671-027ec2e01db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:39.443326 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:39.497607 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 545/1086] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:55:39.540733 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ef105-1c8d-48ee-a031-f6095ca7f933 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:39.625836 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:39.709848 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ef105-1c8d-48ee-a031-f6095ca7f933 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:39.750890 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a4d39-141b-48f7-aefe-6c20a5ad797a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:39.751563 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a4d39-141b-48f7-aefe-6c20a5ad797a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:39.751850 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a4d39-141b-48f7-aefe-6c20a5ad797a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:39.752123 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a4d39-141b-48f7-aefe-6c20a5ad797a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:39.766459 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a4d39-141b-48f7-aefe-6c20a5ad797a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:39.771541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-431a4d39-141b-48f7-aefe-6c20a5ad797a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:55:39.771541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:39.771541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:39.771541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:39.771541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:39.771541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:39.771541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:39.771541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:39.771541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:39.771541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:39.771541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:39.771541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:39.771541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:39.771541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:39.771541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:39.771541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:39.779057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:39.779057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:39.779057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:39.779057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:39.779057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:39.779057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:39.779057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:39.779057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:39.779057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:39.779057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:39.779057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:39.779057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:39.779057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:39.779057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:39.779057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:39.779057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:39.780677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:39.780677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:39.780677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:39.780677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:39.780677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:39.780677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:39.780677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:39.780677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:39.780677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:39.780677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:39.780677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:55:39.780677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:39.780677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 542/1087] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:55:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:39.783575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98ef27-2690-4f5a-a671-027ec2e01db1 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:39.907177 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98ef27-2690-4f5a-a671-027ec2e01db1 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:39.955981 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad390a9-2e4e-485b-b37c-9aeedd06420f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:39.956661 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad390a9-2e4e-485b-b37c-9aeedd06420f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:39.956939 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad390a9-2e4e-485b-b37c-9aeedd06420f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:39.957213 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad390a9-2e4e-485b-b37c-9aeedd06420f None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:39.957951 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad390a9-2e4e-485b-b37c-9aeedd06420f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:39.958250 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad390a9-2e4e-485b-b37c-9aeedd06420f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:39.959728 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad390a9-2e4e-485b-b37c-9aeedd06420f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:39.969597 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 546/1088] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:39.975643 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a4d39-141b-48f7-aefe-6c20a5ad797a None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:40.003190 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10784fe0-e5f2-4bb9-b842-93622d71146f None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:40.003572 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10784fe0-e5f2-4bb9-b842-93622d71146f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:40.003867 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10784fe0-e5f2-4bb9-b842-93622d71146f None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:40.005356 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 547/1089] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:55:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:40.014143 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf9c7f-665f-4cdd-9823-41a0934f4fce None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:40.014640 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf9c7f-665f-4cdd-9823-41a0934f4fce None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:40.015093 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf9c7f-665f-4cdd-9823-41a0934f4fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:40.037234 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b7cf9c7f-665f-4cdd-9823-41a0934f4fce None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:40.038462 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b7cf9c7f-665f-4cdd-9823-41a0934f4fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:40.089743 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:40.127943 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a4d39-141b-48f7-aefe-6c20a5ad797a None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:40.130512 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:40.155213 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 548/1090] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:55:40.171838 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eecce13-6f03-4bc6-91a8-0f71746da796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:40.172483 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eecce13-6f03-4bc6-91a8-0f71746da796 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:40.172771 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eecce13-6f03-4bc6-91a8-0f71746da796 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:40.173050 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eecce13-6f03-4bc6-91a8-0f71746da796 None admin] PATH_INFO: `/v3/users/e8d61e11850b4d8f855a2926ed6c79ce` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:40.174292 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eecce13-6f03-4bc6-91a8-0f71746da796 None admin] RBAC: Authorizing `identity:get_user(user_id=e8d61e11850b4d8f855a2926ed6c79ce)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:40.175785 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eecce13-6f03-4bc6-91a8-0f71746da796 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:40.177549 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 543/1091] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:39 2022] GET /identity/v3/users/e8d61e11850b4d8f855a2926ed6c79ce => generated 285 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:40.185888 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf9c7f-665f-4cdd-9823-41a0934f4fce None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:40.186127 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eecce13-6f03-4bc6-91a8-0f71746da796 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:40.312149 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf9c7f-665f-4cdd-9823-41a0934f4fce None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:40.316750 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eecce13-6f03-4bc6-91a8-0f71746da796 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:40.366103 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9005b7-7798-459b-be56-2c2d33f1d4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:40.366838 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9005b7-7798-459b-be56-2c2d33f1d4dc None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:40.367164 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9005b7-7798-459b-be56-2c2d33f1d4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:40.367357 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9005b7-7798-459b-be56-2c2d33f1d4dc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:40.375071 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1044bb-76e2-410e-bed8-06ee2db4e40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:40.376180 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1044bb-76e2-410e-bed8-06ee2db4e40e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:40.376548 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1044bb-76e2-410e-bed8-06ee2db4e40e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:40.377040 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1044bb-76e2-410e-bed8-06ee2db4e40e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:40.381083 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9005b7-7798-459b-be56-2c2d33f1d4dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:40.385405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-ab9005b7-7798-459b-be56-2c2d33f1d4dc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 13:55:40.385405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:40.385405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:40.385405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:40.385405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:40.385405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:40.385405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:40.385405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:40.385405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:40.385405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:40.385405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:40.385405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:40.385405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:40.385405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:40.385405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:40.385405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:40.386611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:40.386611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:40.386611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:40.386611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:40.386611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:40.386611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:40.386611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:40.386611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:40.386611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:40.386611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:40.386611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:40.386611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:40.386611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:40.386611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:40.386611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:40.386611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:40.387475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:40.387475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:40.387475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:40.387475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:40.387475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:40.387475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:40.387475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:40.387475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:40.387475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:40.387475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:40.387475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 13:55:40.387475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:40.387475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 549/1092] 10.222.0.45 () {56 vars in 1120 bytes} [Mon Aug 1 13:55:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:40.393247 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9005b7-7798-459b-be56-2c2d33f1d4dc None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:40.423720 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:40.457716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:40.501807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1044bb-76e2-410e-bed8-06ee2db4e40e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:40.504025 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1044bb-76e2-410e-bed8-06ee2db4e40e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:40.531079 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9005b7-7798-459b-be56-2c2d33f1d4dc None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:40.584061 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd2cab5-f0d5-4780-8fdb-e0e65b93084c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:40.584738 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2cab5-f0d5-4780-8fdb-e0e65b93084c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:40.585016 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2cab5-f0d5-4780-8fdb-e0e65b93084c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:40.585264 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2cab5-f0d5-4780-8fdb-e0e65b93084c None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:40.586002 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd2cab5-f0d5-4780-8fdb-e0e65b93084c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:40.586306 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd2cab5-f0d5-4780-8fdb-e0e65b93084c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:40.587719 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd2cab5-f0d5-4780-8fdb-e0e65b93084c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:40.597300 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 550/1093] 10.222.0.45 () {56 vars in 1115 bytes} [Mon Aug 1 13:55:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:40.603439 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd2cab5-f0d5-4780-8fdb-e0e65b93084c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:40.642287 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 544/1094] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:55:40.648989 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1044bb-76e2-410e-bed8-06ee2db4e40e None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:40.735568 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:40.738688 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:40.769751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd2cab5-f0d5-4780-8fdb-e0e65b93084c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:40.776617 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1044bb-76e2-410e-bed8-06ee2db4e40e None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:40.813865 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb1fa1-421c-4692-bd85-f3667251ef99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:40.814673 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb1fa1-421c-4692-bd85-f3667251ef99 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:40.815060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb1fa1-421c-4692-bd85-f3667251ef99 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:40.815476 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb1fa1-421c-4692-bd85-f3667251ef99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:40.822528 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932db47f-ed54-4940-9afe-54dbe2ff1368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:40.823219 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932db47f-ed54-4940-9afe-54dbe2ff1368 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:40.823499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932db47f-ed54-4940-9afe-54dbe2ff1368 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:40.823770 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932db47f-ed54-4940-9afe-54dbe2ff1368 None admin] PATH_INFO: `/v3/projects/e9050cf3677442a6a5d5bf54ad0dc036` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:40.825263 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932db47f-ed54-4940-9afe-54dbe2ff1368 None admin] RBAC: Authorizing `identity:get_project(project_id=e9050cf3677442a6a5d5bf54ad0dc036)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:40.826777 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932db47f-ed54-4940-9afe-54dbe2ff1368 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:40.828670 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 545/1095] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:40 2022] GET /identity/v3/projects/e9050cf3677442a6a5d5bf54ad0dc036 => generated 311 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:40.831891 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eb1fa1-421c-4692-bd85-f3667251ef99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:40.833677 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eb1fa1-421c-4692-bd85-f3667251ef99 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:40.838386 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932db47f-ed54-4940-9afe-54dbe2ff1368 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:40.851249 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-a5eb1fa1-421c-4692-bd85-f3667251ef99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:55:40.851249 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:40.851249 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:40.851249 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:40.851249 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:40.851249 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:40.851249 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:40.851249 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:40.851249 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:40.851249 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:40.851249 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:40.851249 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:40.851249 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:55:40.851249 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:55:40.851249 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:55:40.852317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:55:40.852317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:40.852317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:40.852317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:40.852317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:40.852317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:40.852317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:40.852317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:40.852317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:40.852317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:40.852317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:40.852317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:40.852317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:40.852317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:40.852317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:40.852317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:40.852317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:40.853868 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:55:40.853868 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:55:40.853868 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:55:40.853868 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:40.853868 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 551/1096] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:55:40.861628 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb1fa1-421c-4692-bd85-f3667251ef99 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:41.025964 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932db47f-ed54-4940-9afe-54dbe2ff1368 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:41.032918 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb1fa1-421c-4692-bd85-f3667251ef99 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:41.081016 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:41.088991 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45fd2d5-467d-41ef-b694-4d39e62af22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:41.089840 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45fd2d5-467d-41ef-b694-4d39e62af22e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:41.090345 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45fd2d5-467d-41ef-b694-4d39e62af22e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:41.090637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45fd2d5-467d-41ef-b694-4d39e62af22e None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:41.091537 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45fd2d5-467d-41ef-b694-4d39e62af22e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:41.091849 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45fd2d5-467d-41ef-b694-4d39e62af22e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:41.093772 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45fd2d5-467d-41ef-b694-4d39e62af22e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:41.096874 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dbb3ba-c094-48cd-a374-096cd8bbbfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:41.097985 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbb3ba-c094-48cd-a374-096cd8bbbfc9 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:41.098456 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbb3ba-c094-48cd-a374-096cd8bbbfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:41.098933 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbb3ba-c094-48cd-a374-096cd8bbbfc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:41.100538 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dbb3ba-c094-48cd-a374-096cd8bbbfc9 None admin] RBAC: Adding query filter params (role.id=3e1a9ff016c840c5b51ef7e3da9900ac, scope.project.id=e9050cf3677442a6a5d5bf54ad0dc036, user.id=e8d61e11850b4d8f855a2926ed6c79ce) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:41.101101 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dbb3ba-c094-48cd-a374-096cd8bbbfc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:41.103921 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dbb3ba-c094-48cd-a374-096cd8bbbfc9 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:41.109131 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 552/1097] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:55:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:41.116768 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 546/1098] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:55:40 2022] GET /identity/v3/role_assignments?user.id=e8d61e11850b4d8f855a2926ed6c79ce&scope.project.id=e9050cf3677442a6a5d5bf54ad0dc036&role.id=3e1a9ff016c840c5b51ef7e3da9900ac => generated 608 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:41.117822 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45fd2d5-467d-41ef-b694-4d39e62af22e None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:41.248781 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45fd2d5-467d-41ef-b694-4d39e62af22e None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:41.304857 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb110c-5479-4c71-a7bd-b1677f816404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:41.305440 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb110c-5479-4c71-a7bd-b1677f816404 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:41.305724 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb110c-5479-4c71-a7bd-b1677f816404 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:41.306012 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb110c-5479-4c71-a7bd-b1677f816404 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:41.315605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb110c-5479-4c71-a7bd-b1677f816404 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:41.317352 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb110c-5479-4c71-a7bd-b1677f816404 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:41.331973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-1adb110c-5479-4c71-a7bd-b1677f816404 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:55:41.331973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:41.331973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:41.331973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:41.331973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:41.331973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:41.331973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:41.331973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:41.331973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:41.331973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:41.331973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:41.331973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:41.331973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:55:41.331973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:55:41.331973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:55:41.331973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:55:41.333916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:41.333916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:41.333916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:55:41.333916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:55:41.333916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:55:41.333916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:55:41.333916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:41.333916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:41.333916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:41.333916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:41.333916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:41.333916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:41.333916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:41.333916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:41.333916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:41.333916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:41.333916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:41.333916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:41.335749 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:41.335749 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:41.335749 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:55:41.335749 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:55:41.335749 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:55:41.335749 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:55:41.335749 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:55:41.335749 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:55:41.335749 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:55:41.335749 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:55:41.335749 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 553/1099] 10.222.0.45 () {56 vars in 1093 bytes} [Mon Aug 1 13:55:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:55:41.342476 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dbb3ba-c094-48cd-a374-096cd8bbbfc9 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:41.348748 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:41.469118 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dbb3ba-c094-48cd-a374-096cd8bbbfc9 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:41.508354 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c742b8-3a6f-4a3f-84ce-7884dbe7032e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:41.509020 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c742b8-3a6f-4a3f-84ce-7884dbe7032e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:41.509299 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c742b8-3a6f-4a3f-84ce-7884dbe7032e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:41.509569 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c742b8-3a6f-4a3f-84ce-7884dbe7032e None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:41.510394 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c742b8-3a6f-4a3f-84ce-7884dbe7032e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:41.510700 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c742b8-3a6f-4a3f-84ce-7884dbe7032e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:41.512204 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c742b8-3a6f-4a3f-84ce-7884dbe7032e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:41.533030 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 547/1100] 10.222.0.45 () {56 vars in 1133 bytes} [Mon Aug 1 13:55:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:41.539736 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb110c-5479-4c71-a7bd-b1677f816404 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:41.613126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:41.671967 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb110c-5479-4c71-a7bd-b1677f816404 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:41.716289 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f994971-9837-4be7-90a4-d95de8013f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:41.717029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f994971-9837-4be7-90a4-d95de8013f74 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:41.717313 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f994971-9837-4be7-90a4-d95de8013f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:41.717585 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f994971-9837-4be7-90a4-d95de8013f74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:41.799853 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f994971-9837-4be7-90a4-d95de8013f74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:41.801398 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f994971-9837-4be7-90a4-d95de8013f74 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:41.860189 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:41.929082 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 554/1101] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:55:41.935940 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c742b8-3a6f-4a3f-84ce-7884dbe7032e None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:42.053922 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:42.116687 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c742b8-3a6f-4a3f-84ce-7884dbe7032e None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:42.182171 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853233c6-c0df-4155-9b85-946639b6767a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:42.183212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853233c6-c0df-4155-9b85-946639b6767a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:42.183689 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853233c6-c0df-4155-9b85-946639b6767a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:42.184143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853233c6-c0df-4155-9b85-946639b6767a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:42.202228 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853233c6-c0df-4155-9b85-946639b6767a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:42.205083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853233c6-c0df-4155-9b85-946639b6767a None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:42.234938 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-853233c6-c0df-4155-9b85-946639b6767a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:55:42.234938 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:42.234938 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:42.234938 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:42.234938 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:42.234938 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:42.234938 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:42.234938 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:42.234938 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:42.234938 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:42.234938 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:42.234938 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:42.234938 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:55:42.234938 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:55:42.234938 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:55:42.236256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:55:42.236256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:42.236256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:42.236256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:42.236256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:42.236256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:42.236256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:42.236256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:42.236256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:42.236256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:42.236256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:42.236256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:42.236256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:42.236256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:42.236256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:42.236256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:42.236256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:42.238034 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:55:42.238034 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:55:42.238034 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:55:42.238034 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:42.238034 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 548/1102] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:42.243331 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f994971-9837-4be7-90a4-d95de8013f74 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:42.393017 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f994971-9837-4be7-90a4-d95de8013f74 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:42.463565 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1351e79-ada8-45a4-94bd-05be813e4ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:42.464641 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1351e79-ada8-45a4-94bd-05be813e4ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:42.465143 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1351e79-ada8-45a4-94bd-05be813e4ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:42.465635 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1351e79-ada8-45a4-94bd-05be813e4ec1 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:42.466985 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1351e79-ada8-45a4-94bd-05be813e4ec1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:42.467536 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1351e79-ada8-45a4-94bd-05be813e4ec1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:42.470343 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1351e79-ada8-45a4-94bd-05be813e4ec1 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:42.494263 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 555/1103] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:55:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:42.503116 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853233c6-c0df-4155-9b85-946639b6767a None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:42.684165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853233c6-c0df-4155-9b85-946639b6767a None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:42.708205 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:42.749710 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e1f508-c67b-424a-a9cc-28c96d8064ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:42.750543 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1f508-c67b-424a-a9cc-28c96d8064ea None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:42.750979 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1f508-c67b-424a-a9cc-28c96d8064ea None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:42.751368 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1f508-c67b-424a-a9cc-28c96d8064ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:42.763350 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e1f508-c67b-424a-a9cc-28c96d8064ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:42.763350 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e1f508-c67b-424a-a9cc-28c96d8064ea None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:42.786234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-57e1f508-c67b-424a-a9cc-28c96d8064ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:55:42.786234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:42.786234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:42.786234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:42.786234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:42.786234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:42.786234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:42.786234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:42.786234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:42.786234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:42.786234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:42.786234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:42.786234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:55:42.786234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:55:42.786234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:55:42.787652 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:55:42.787652 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:42.787652 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:42.787652 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:42.787652 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:42.787652 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:42.787652 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:42.787652 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:42.787652 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:42.787652 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:42.787652 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:42.787652 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:42.787652 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:42.787652 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:42.787652 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:42.787652 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:42.787652 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:42.788628 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:55:42.788628 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:55:42.788628 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:55:42.788628 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:55:42.788628 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:55:42.788628 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:55:42.788628 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:55:42.788628 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:55:42.788628 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 549/1104] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:55:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:42.798732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1351e79-ada8-45a4-94bd-05be813e4ec1 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:42.826873 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:42.988203 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1351e79-ada8-45a4-94bd-05be813e4ec1 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:43.055155 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f35fe1e-3c6e-41ae-aad2-5c314856bac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:43.056156 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f35fe1e-3c6e-41ae-aad2-5c314856bac9 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:43.056713 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f35fe1e-3c6e-41ae-aad2-5c314856bac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:43.057217 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f35fe1e-3c6e-41ae-aad2-5c314856bac9 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:43.058730 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f35fe1e-3c6e-41ae-aad2-5c314856bac9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:43.059302 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f35fe1e-3c6e-41ae-aad2-5c314856bac9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:43.062134 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f35fe1e-3c6e-41ae-aad2-5c314856bac9 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:43.085073 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 556/1105] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:55:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:43.095390 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e1f508-c67b-424a-a9cc-28c96d8064ea None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:43.273086 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e1f508-c67b-424a-a9cc-28c96d8064ea None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:43.323262 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a4b921-22a7-4f6f-baad-5dcf47a0e5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:43.324006 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a4b921-22a7-4f6f-baad-5dcf47a0e5a4 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:43.324432 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a4b921-22a7-4f6f-baad-5dcf47a0e5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:43.324814 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a4b921-22a7-4f6f-baad-5dcf47a0e5a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:43.325759 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a4b921-22a7-4f6f-baad-5dcf47a0e5a4 None admin] RBAC: Adding query filter params (role.id=c29b2bae1d6c4e438cb861ed195d5c72, scope.project.id=36f063d8f65d494a89dbbd72c53417c8, user.id=5b3adc14dcaf4ac79b43bb97fa1ba1d4) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:43.326180 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a4b921-22a7-4f6f-baad-5dcf47a0e5a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:43.327777 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a4b921-22a7-4f6f-baad-5dcf47a0e5a4 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:43.336409 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 550/1106] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:55:43 2022] GET /identity/v3/role_assignments?user.id=5b3adc14dcaf4ac79b43bb97fa1ba1d4&scope.project.id=36f063d8f65d494a89dbbd72c53417c8&role.id=c29b2bae1d6c4e438cb861ed195d5c72 => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:43.990318 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b70c6c-c074-4ad7-afd2-90589afa6260 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:43.991082 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b70c6c-c074-4ad7-afd2-90589afa6260 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:43.991082 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b70c6c-c074-4ad7-afd2-90589afa6260 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:43.992384 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 557/1107] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:43.999515 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85348b-2521-461d-b9ae-3ff1d028bd8b None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:44.000500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85348b-2521-461d-b9ae-3ff1d028bd8b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:44.000642 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85348b-2521-461d-b9ae-3ff1d028bd8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:44.025450 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-5b85348b-2521-461d-b9ae-3ff1d028bd8b None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:44.027568 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5b85348b-2521-461d-b9ae-3ff1d028bd8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:44.079533 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:44.141138 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 551/1108] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:55:44.170601 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b70c6c-c074-4ad7-afd2-90589afa6260 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:44.290498 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:44.339001 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b70c6c-c074-4ad7-afd2-90589afa6260 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:44.394678 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca65e0a-3159-4745-9673-a85aa37e4126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:44.395365 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca65e0a-3159-4745-9673-a85aa37e4126 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:44.395642 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca65e0a-3159-4745-9673-a85aa37e4126 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:44.395909 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca65e0a-3159-4745-9673-a85aa37e4126 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:44.397742 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca65e0a-3159-4745-9673-a85aa37e4126 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:44.399301 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca65e0a-3159-4745-9673-a85aa37e4126 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:44.401309 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 558/1109] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:44.411583 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b85348b-2521-461d-b9ae-3ff1d028bd8b None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:44.590427 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b85348b-2521-461d-b9ae-3ff1d028bd8b None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:44.647327 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d928862b-96ac-446b-a335-256c1a9adb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:44.648065 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928862b-96ac-446b-a335-256c1a9adb84 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:44.648480 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928862b-96ac-446b-a335-256c1a9adb84 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:44.648806 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928862b-96ac-446b-a335-256c1a9adb84 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:44.649602 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d928862b-96ac-446b-a335-256c1a9adb84 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:44.651178 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d928862b-96ac-446b-a335-256c1a9adb84 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:44.680678 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 552/1110] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 13:55:44 2022] POST /identity/v3/users => generated 286 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:55:47.108967 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1227d04-17b4-472a-9559-b62bba36c757 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:47.109749 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1227d04-17b4-472a-9559-b62bba36c757 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:47.109749 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1227d04-17b4-472a-9559-b62bba36c757 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:47.111010 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 559/1111] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:55:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:55:47.119328 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4632c-6438-44b4-9675-bc262ff72454 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:47.120116 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4632c-6438-44b4-9675-bc262ff72454 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:47.120116 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4632c-6438-44b4-9675-bc262ff72454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:47.142372 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-49a4632c-6438-44b4-9675-bc262ff72454 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:47.143773 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-49a4632c-6438-44b4-9675-bc262ff72454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:47.174950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:47.246722 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 553/1112] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:55:47.271768 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1227d04-17b4-472a-9559-b62bba36c757 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:47.453819 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1227d04-17b4-472a-9559-b62bba36c757 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:47.505797 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350affe8-2089-467d-938d-123188e015fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:47.506475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350affe8-2089-467d-938d-123188e015fe None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:47.506754 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350affe8-2089-467d-938d-123188e015fe None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:47.507030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350affe8-2089-467d-938d-123188e015fe None admin] PATH_INFO: `/v3/roles/60efb70fa7354ebfb04aa5e9bdad2a78` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:47.509070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350affe8-2089-467d-938d-123188e015fe None admin] RBAC: Authorizing `identity:get_role(role_id=60efb70fa7354ebfb04aa5e9bdad2a78)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:47.511634 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350affe8-2089-467d-938d-123188e015fe None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:47.513463 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 560/1113] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:47 2022] GET /identity/v3/roles/60efb70fa7354ebfb04aa5e9bdad2a78 => generated 223 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:47.520498 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4632c-6438-44b4-9675-bc262ff72454 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:47.666002 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4632c-6438-44b4-9675-bc262ff72454 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:47.721302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e4f7a-7e73-42a0-9a18-d2da967e7355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:47.722159 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e4f7a-7e73-42a0-9a18-d2da967e7355 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:47.722543 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e4f7a-7e73-42a0-9a18-d2da967e7355 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:47.722997 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e4f7a-7e73-42a0-9a18-d2da967e7355 None admin] PATH_INFO: `/v3/users/017f1b8608b648d59b4e60c61801d37d` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:47.746609 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e4f7a-7e73-42a0-9a18-d2da967e7355 None admin] RBAC: Authorizing `identity:get_user(user_id=017f1b8608b648d59b4e60c61801d37d)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:47.749038 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e4f7a-7e73-42a0-9a18-d2da967e7355 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:47.752141 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 554/1114] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:47 2022] GET /identity/v3/users/017f1b8608b648d59b4e60c61801d37d => generated 286 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:47.762392 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350affe8-2089-467d-938d-123188e015fe None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:47.784474 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:47.937452 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350affe8-2089-467d-938d-123188e015fe None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:47.990635 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3828c419-dabf-4e2f-85bb-5e497a240789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:47.990964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3828c419-dabf-4e2f-85bb-5e497a240789 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:47.991403 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3828c419-dabf-4e2f-85bb-5e497a240789 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:47.991881 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3828c419-dabf-4e2f-85bb-5e497a240789 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:48.096853 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3828c419-dabf-4e2f-85bb-5e497a240789 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:48.098435 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3828c419-dabf-4e2f-85bb-5e497a240789 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:48.112255 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:48.235840 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 561/1115] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:55:48.243197 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e4f7a-7e73-42a0-9a18-d2da967e7355 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:48.265301 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:48.443150 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e4f7a-7e73-42a0-9a18-d2da967e7355 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:48.515243 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a1411-c0db-4f87-b50a-aa7dc011cc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:48.516332 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a1411-c0db-4f87-b50a-aa7dc011cc8a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:48.516823 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a1411-c0db-4f87-b50a-aa7dc011cc8a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:48.517309 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a1411-c0db-4f87-b50a-aa7dc011cc8a None admin] PATH_INFO: `/v3/projects/e9050cf3677442a6a5d5bf54ad0dc036` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:48.519648 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a1411-c0db-4f87-b50a-aa7dc011cc8a None admin] RBAC: Authorizing `identity:get_project(project_id=e9050cf3677442a6a5d5bf54ad0dc036)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:48.522857 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a1411-c0db-4f87-b50a-aa7dc011cc8a None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:48.526285 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 555/1116] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:48 2022] GET /identity/v3/projects/e9050cf3677442a6a5d5bf54ad0dc036 => generated 311 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:48.535581 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3828c419-dabf-4e2f-85bb-5e497a240789 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:48.694285 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3828c419-dabf-4e2f-85bb-5e497a240789 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:48.709359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:48.738052 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e5cf3-1bae-4163-904a-5ef336477ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:48.738841 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e5cf3-1bae-4163-904a-5ef336477ecd None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:48.739212 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e5cf3-1bae-4163-904a-5ef336477ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:48.739578 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e5cf3-1bae-4163-904a-5ef336477ecd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:48.740572 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8e5cf3-1bae-4163-904a-5ef336477ecd None admin] RBAC: Adding query filter params (role.id=60efb70fa7354ebfb04aa5e9bdad2a78, scope.project.id=e9050cf3677442a6a5d5bf54ad0dc036, user.id=017f1b8608b648d59b4e60c61801d37d) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:48.740994 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8e5cf3-1bae-4163-904a-5ef336477ecd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:48.742610 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8e5cf3-1bae-4163-904a-5ef336477ecd None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:48.749329 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 562/1117] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:55:48 2022] GET /identity/v3/role_assignments?user.id=017f1b8608b648d59b4e60c61801d37d&scope.project.id=e9050cf3677442a6a5d5bf54ad0dc036&role.id=60efb70fa7354ebfb04aa5e9bdad2a78 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:50.902004 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ed92e-1e8f-46f4-bdf0-1f32d372c787 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:50.902568 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ed92e-1e8f-46f4-bdf0-1f32d372c787 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:50.902878 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ed92e-1e8f-46f4-bdf0-1f32d372c787 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:50.905504 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 556/1118] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:55:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:50.914499 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd61e7-6d15-494d-a7d7-344dae4723cd None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:50.915363 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd61e7-6d15-494d-a7d7-344dae4723cd None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:50.915363 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd61e7-6d15-494d-a7d7-344dae4723cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:50.939457 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-85fd61e7-6d15-494d-a7d7-344dae4723cd None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:50.940782 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-85fd61e7-6d15-494d-a7d7-344dae4723cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:51.028290 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 563/1119] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:55:51.052377 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ed92e-1e8f-46f4-bdf0-1f32d372c787 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:51.167130 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:51.193971 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ed92e-1e8f-46f4-bdf0-1f32d372c787 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:51.247183 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d2afef-fab6-4623-89da-116435778c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:51.247882 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d2afef-fab6-4623-89da-116435778c06 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:51.248143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d2afef-fab6-4623-89da-116435778c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:51.248444 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d2afef-fab6-4623-89da-116435778c06 None admin] PATH_INFO: `/v3/roles/60efb70fa7354ebfb04aa5e9bdad2a78` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:51.250240 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d2afef-fab6-4623-89da-116435778c06 None admin] RBAC: Authorizing `identity:get_role(role_id=60efb70fa7354ebfb04aa5e9bdad2a78)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:51.251867 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d2afef-fab6-4623-89da-116435778c06 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:51.253478 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 557/1120] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:51 2022] GET /identity/v3/roles/60efb70fa7354ebfb04aa5e9bdad2a78 => generated 223 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:51.260119 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd61e7-6d15-494d-a7d7-344dae4723cd None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:51.412053 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd61e7-6d15-494d-a7d7-344dae4723cd None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:51.415054 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:51.468302 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785dc14-677a-415f-bb7d-96b55030d6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:51.469292 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785dc14-677a-415f-bb7d-96b55030d6a3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:51.469775 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785dc14-677a-415f-bb7d-96b55030d6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:51.470232 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785dc14-677a-415f-bb7d-96b55030d6a3 None admin] PATH_INFO: `/v3/users/017f1b8608b648d59b4e60c61801d37d` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:51.472193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785dc14-677a-415f-bb7d-96b55030d6a3 None admin] RBAC: Authorizing `identity:get_user(user_id=017f1b8608b648d59b4e60c61801d37d)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:51.474644 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785dc14-677a-415f-bb7d-96b55030d6a3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:51.477225 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 564/1121] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:51 2022] GET /identity/v3/users/017f1b8608b648d59b4e60c61801d37d => generated 286 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:51.486819 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d2afef-fab6-4623-89da-116435778c06 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:51.608933 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d2afef-fab6-4623-89da-116435778c06 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:51.659952 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3841542-ab3e-410d-ad24-27aba253f0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:51.661017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3841542-ab3e-410d-ad24-27aba253f0ca None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:51.661492 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3841542-ab3e-410d-ad24-27aba253f0ca None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:51.661935 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3841542-ab3e-410d-ad24-27aba253f0ca None admin] PATH_INFO: `/v3/projects/e9050cf3677442a6a5d5bf54ad0dc036` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:51.664077 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3841542-ab3e-410d-ad24-27aba253f0ca None admin] RBAC: Authorizing `identity:get_project(project_id=e9050cf3677442a6a5d5bf54ad0dc036)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:51.666816 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3841542-ab3e-410d-ad24-27aba253f0ca None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:51.670104 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 558/1122] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:51 2022] GET /identity/v3/projects/e9050cf3677442a6a5d5bf54ad0dc036 => generated 311 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:51.680903 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785dc14-677a-415f-bb7d-96b55030d6a3 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:51.846509 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785dc14-677a-415f-bb7d-96b55030d6a3 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:51.886013 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec6932-a823-42a4-a81a-a97ca371d038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:51.886755 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec6932-a823-42a4-a81a-a97ca371d038 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:51.887072 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec6932-a823-42a4-a81a-a97ca371d038 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:51.887388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec6932-a823-42a4-a81a-a97ca371d038 None admin] PATH_INFO: `/v3/projects/e9050cf3677442a6a5d5bf54ad0dc036/users/017f1b8608b648d59b4e60c61801d37d/roles/60efb70fa7354ebfb04aa5e9bdad2a78` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:51.889816 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ec6932-a823-42a4-a81a-a97ca371d038 None admin] RBAC: Authorizing `identity:create_grant(project_id=e9050cf3677442a6a5d5bf54ad0dc036, user_id=017f1b8608b648d59b4e60c61801d37d, role_id=60efb70fa7354ebfb04aa5e9bdad2a78)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:51.891468 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ec6932-a823-42a4-a81a-a97ca371d038 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:51.909543 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 565/1123] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 13:55:51 2022] PUT /identity/v3/projects/e9050cf3677442a6a5d5bf54ad0dc036/users/017f1b8608b648d59b4e60c61801d37d/roles/60efb70fa7354ebfb04aa5e9bdad2a78 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:55:54.226641 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33b13b-d6c3-4d71-9899-9bb24fbfc008 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:54.227386 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33b13b-d6c3-4d71-9899-9bb24fbfc008 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:54.227515 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33b13b-d6c3-4d71-9899-9bb24fbfc008 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:54.229048 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 559/1124] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:55:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:54.235392 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2fbbdf-3da0-4534-ad25-7a4a3d7c3e78 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:54.236153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2fbbdf-3da0-4534-ad25-7a4a3d7c3e78 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:54.236153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2fbbdf-3da0-4534-ad25-7a4a3d7c3e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:54.255859 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-5e2fbbdf-3da0-4534-ad25-7a4a3d7c3e78 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:54.257129 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5e2fbbdf-3da0-4534-ad25-7a4a3d7c3e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:54.284427 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:54.335434 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 566/1125] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:55:54.363419 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa33b13b-d6c3-4d71-9899-9bb24fbfc008 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:54.393206 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:54.521970 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa33b13b-d6c3-4d71-9899-9bb24fbfc008 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:54.576332 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb1719-7ee8-4245-9486-bf02c482a889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:54.577461 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb1719-7ee8-4245-9486-bf02c482a889 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:54.577959 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb1719-7ee8-4245-9486-bf02c482a889 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:54.578487 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb1719-7ee8-4245-9486-bf02c482a889 None admin] PATH_INFO: `/v3/roles/60efb70fa7354ebfb04aa5e9bdad2a78` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:54.581169 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb1719-7ee8-4245-9486-bf02c482a889 None admin] RBAC: Authorizing `identity:get_role(role_id=60efb70fa7354ebfb04aa5e9bdad2a78)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:54.583784 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb1719-7ee8-4245-9486-bf02c482a889 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:54.586591 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 560/1126] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:54 2022] GET /identity/v3/roles/60efb70fa7354ebfb04aa5e9bdad2a78 => generated 223 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:54.596063 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2fbbdf-3da0-4534-ad25-7a4a3d7c3e78 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:54.722767 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2fbbdf-3da0-4534-ad25-7a4a3d7c3e78 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:54.784075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a9ebc-390b-4c4e-be2a-fe95442a4244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:54.785118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a9ebc-390b-4c4e-be2a-fe95442a4244 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:54.785528 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a9ebc-390b-4c4e-be2a-fe95442a4244 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:54.786014 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a9ebc-390b-4c4e-be2a-fe95442a4244 None admin] PATH_INFO: `/v3/users/017f1b8608b648d59b4e60c61801d37d` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:54.788142 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a9ebc-390b-4c4e-be2a-fe95442a4244 None admin] RBAC: Authorizing `identity:get_user(user_id=017f1b8608b648d59b4e60c61801d37d)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:54.790735 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a9ebc-390b-4c4e-be2a-fe95442a4244 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:54.793849 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 567/1127] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:54 2022] GET /identity/v3/users/017f1b8608b648d59b4e60c61801d37d => generated 286 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:54.802885 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb1719-7ee8-4245-9486-bf02c482a889 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:54.937718 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:54.961496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb1719-7ee8-4245-9486-bf02c482a889 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:55.006060 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bff7a2-2d8f-4914-964d-4bd715fb17d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:55.006775 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bff7a2-2d8f-4914-964d-4bd715fb17d4 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:55.007097 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bff7a2-2d8f-4914-964d-4bd715fb17d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:55.007416 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bff7a2-2d8f-4914-964d-4bd715fb17d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:55.095241 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bff7a2-2d8f-4914-964d-4bd715fb17d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:55.097848 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bff7a2-2d8f-4914-964d-4bd715fb17d4 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:55.199431 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:55.221859 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 561/1128] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:55:55.230944 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a9ebc-390b-4c4e-be2a-fe95442a4244 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:55.271584 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:55.363754 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a9ebc-390b-4c4e-be2a-fe95442a4244 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:55.403899 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7bc5a-931e-415d-b19e-6e2b60f2e162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:55.404589 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7bc5a-931e-415d-b19e-6e2b60f2e162 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:55.404921 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7bc5a-931e-415d-b19e-6e2b60f2e162 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:55.405195 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7bc5a-931e-415d-b19e-6e2b60f2e162 None admin] PATH_INFO: `/v3/projects/e9050cf3677442a6a5d5bf54ad0dc036` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:55.406500 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7bc5a-931e-415d-b19e-6e2b60f2e162 None admin] RBAC: Authorizing `identity:get_project(project_id=e9050cf3677442a6a5d5bf54ad0dc036)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:55.408111 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7bc5a-931e-415d-b19e-6e2b60f2e162 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:55.409929 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 568/1129] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:55 2022] GET /identity/v3/projects/e9050cf3677442a6a5d5bf54ad0dc036 => generated 311 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:55.416638 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bff7a2-2d8f-4914-964d-4bd715fb17d4 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:55.533289 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bff7a2-2d8f-4914-964d-4bd715fb17d4 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:55.589497 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9cd2a5-e251-4331-a188-a87745292c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:55.590507 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cd2a5-e251-4331-a188-a87745292c07 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:55.590981 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cd2a5-e251-4331-a188-a87745292c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:55.591451 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cd2a5-e251-4331-a188-a87745292c07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:55.595266 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9cd2a5-e251-4331-a188-a87745292c07 None admin] RBAC: Adding query filter params (role.id=60efb70fa7354ebfb04aa5e9bdad2a78, scope.project.id=e9050cf3677442a6a5d5bf54ad0dc036, user.id=017f1b8608b648d59b4e60c61801d37d) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:55.595672 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9cd2a5-e251-4331-a188-a87745292c07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:55.597690 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9cd2a5-e251-4331-a188-a87745292c07 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:55.613530 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 562/1130] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:55:55 2022] GET /identity/v3/role_assignments?user.id=017f1b8608b648d59b4e60c61801d37d&scope.project.id=e9050cf3677442a6a5d5bf54ad0dc036&role.id=60efb70fa7354ebfb04aa5e9bdad2a78 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:57.924332 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06771022-8be9-46ad-aab5-46a691aabb6c None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:57.925086 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06771022-8be9-46ad-aab5-46a691aabb6c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:57.925086 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06771022-8be9-46ad-aab5-46a691aabb6c None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:57.926441 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 569/1131] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:55:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:57.932517 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f875ced-da67-47a5-a785-471748c7d509 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:57.933264 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f875ced-da67-47a5-a785-471748c7d509 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:57.933264 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f875ced-da67-47a5-a785-471748c7d509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:57.953013 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-1f875ced-da67-47a5-a785-471748c7d509 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:57.954448 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1f875ced-da67-47a5-a785-471748c7d509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:58.005266 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:58.039338 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 563/1132] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:55:58.080382 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06771022-8be9-46ad-aab5-46a691aabb6c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:58.200830 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:58.209369 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06771022-8be9-46ad-aab5-46a691aabb6c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:58.253730 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb3eab-bc61-4db0-af21-7977dd3050af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:58.254384 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb3eab-bc61-4db0-af21-7977dd3050af None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:58.254658 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb3eab-bc61-4db0-af21-7977dd3050af None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:58.254929 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb3eab-bc61-4db0-af21-7977dd3050af None admin] PATH_INFO: `/v3/domains/default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:58.256153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb3eab-bc61-4db0-af21-7977dd3050af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:58.258517 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb3eab-bc61-4db0-af21-7977dd3050af None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:58.261113 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 570/1133] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:58.270629 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f875ced-da67-47a5-a785-471748c7d509 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:58.404290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f875ced-da67-47a5-a785-471748c7d509 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:58.450150 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd12b6-a22e-4ecf-b41a-a7d32d81f193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:58.450954 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd12b6-a22e-4ecf-b41a-a7d32d81f193 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:58.451245 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd12b6-a22e-4ecf-b41a-a7d32d81f193 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:58.451523 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd12b6-a22e-4ecf-b41a-a7d32d81f193 None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:58.452315 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acd12b6-a22e-4ecf-b41a-a7d32d81f193 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:58.454342 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acd12b6-a22e-4ecf-b41a-a7d32d81f193 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:58.497559 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 564/1134] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Aug 1 13:55:58 2022] POST /identity/v3/projects => generated 311 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:56:00.715749 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c9b227-4af2-4c41-a2a4-11e5504e161b None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:00.716398 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c9b227-4af2-4c41-a2a4-11e5504e161b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:00.716739 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c9b227-4af2-4c41-a2a4-11e5504e161b None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:00.719251 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 571/1135] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:56:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:00.727881 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca6458-106f-44a1-b610-27b4d4174b3f None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:00.728723 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca6458-106f-44a1-b610-27b4d4174b3f None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:00.728723 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca6458-106f-44a1-b610-27b4d4174b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:00.749977 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-42ca6458-106f-44a1-b610-27b4d4174b3f None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:00.751382 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-42ca6458-106f-44a1-b610-27b4d4174b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:00.785214 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:00.844841 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 565/1136] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:56:00.875432 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c9b227-4af2-4c41-a2a4-11e5504e161b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:01.034157 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c9b227-4af2-4c41-a2a4-11e5504e161b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:01.079466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57125bf1-8514-4038-b218-c88c9a04247d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:01.080161 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57125bf1-8514-4038-b218-c88c9a04247d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:01.080589 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57125bf1-8514-4038-b218-c88c9a04247d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:01.080867 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57125bf1-8514-4038-b218-c88c9a04247d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:01.082560 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57125bf1-8514-4038-b218-c88c9a04247d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:01.084755 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57125bf1-8514-4038-b218-c88c9a04247d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:01.087490 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 572/1137] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:01.098088 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca6458-106f-44a1-b610-27b4d4174b3f None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:01.282479 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca6458-106f-44a1-b610-27b4d4174b3f None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:01.322103 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe042c4-089c-4fe5-934b-7476478aa5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:01.322747 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe042c4-089c-4fe5-934b-7476478aa5db None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:01.323022 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe042c4-089c-4fe5-934b-7476478aa5db None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:01.323289 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe042c4-089c-4fe5-934b-7476478aa5db None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:01.323992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe042c4-089c-4fe5-934b-7476478aa5db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:01.325502 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe042c4-089c-4fe5-934b-7476478aa5db None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:01.355461 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 566/1138] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 13:56:01 2022] POST /identity/v3/users => generated 286 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:56:03.507895 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7185b25-0eb4-41c6-8a51-33bd034f3747 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:03.508584 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7185b25-0eb4-41c6-8a51-33bd034f3747 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:03.508584 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7185b25-0eb4-41c6-8a51-33bd034f3747 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:03.509874 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 573/1139] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:03.516019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd8961-f257-4f0b-80fe-00f20a9d1d5a None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:03.516929 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd8961-f257-4f0b-80fe-00f20a9d1d5a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:03.516929 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd8961-f257-4f0b-80fe-00f20a9d1d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:03.537920 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-93bd8961-f257-4f0b-80fe-00f20a9d1d5a None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:03.539163 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-93bd8961-f257-4f0b-80fe-00f20a9d1d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:03.552540 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:03.651996 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 567/1140] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:56:03.694710 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7185b25-0eb4-41c6-8a51-33bd034f3747 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:03.739873 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:03.841479 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7185b25-0eb4-41c6-8a51-33bd034f3747 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:03.885720 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7dffc3-d1a4-46f6-8371-c351804e6e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:03.886376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7dffc3-d1a4-46f6-8371-c351804e6e86 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:03.886652 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7dffc3-d1a4-46f6-8371-c351804e6e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:03.887002 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7dffc3-d1a4-46f6-8371-c351804e6e86 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:03.900500 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7dffc3-d1a4-46f6-8371-c351804e6e86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:03.905690 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-da7dffc3-d1a4-46f6-8371-c351804e6e86 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:56:03.905690 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:03.905690 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:03.905690 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:03.905690 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:03.905690 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:03.905690 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:03.905690 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:03.905690 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:03.905690 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:03.905690 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:03.905690 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:03.905690 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:03.905690 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:03.905690 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:03.905690 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:03.906920 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:03.906920 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:03.906920 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:03.906920 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:03.906920 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:03.906920 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:03.906920 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:03.906920 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:03.906920 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:03.906920 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:03.906920 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:03.906920 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:03.906920 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:03.906920 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:03.906920 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:03.906920 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:03.908386 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:03.908386 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:03.908386 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:03.908386 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:03.908386 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:03.908386 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:03.908386 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:03.908386 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:03.908386 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:03.908386 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:03.908386 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:56:03.908386 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:56:03.908386 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 574/1141] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:56:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:56:03.915655 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bd8961-f257-4f0b-80fe-00f20a9d1d5a None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:04.047473 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bd8961-f257-4f0b-80fe-00f20a9d1d5a None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:04.088748 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d675dd-72f2-4b1f-94ae-453a3eb83f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:04.089388 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d675dd-72f2-4b1f-94ae-453a3eb83f68 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:04.089670 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d675dd-72f2-4b1f-94ae-453a3eb83f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:04.089945 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d675dd-72f2-4b1f-94ae-453a3eb83f68 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:04.090676 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d675dd-72f2-4b1f-94ae-453a3eb83f68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:04.090982 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d675dd-72f2-4b1f-94ae-453a3eb83f68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:04.092544 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d675dd-72f2-4b1f-94ae-453a3eb83f68 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:04.102211 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 568/1142] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:56:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:04.109549 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7dffc3-d1a4-46f6-8371-c351804e6e86 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:04.228989 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:04.239782 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7dffc3-d1a4-46f6-8371-c351804e6e86 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:04.285077 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5688a81e-98b2-4e8b-a6fa-d71e3fb327e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:04.285737 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688a81e-98b2-4e8b-a6fa-d71e3fb327e4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:04.286012 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688a81e-98b2-4e8b-a6fa-d71e3fb327e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:04.286284 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688a81e-98b2-4e8b-a6fa-d71e3fb327e4 None admin] PATH_INFO: `/v3/users/65cde5484e14423082cc24d75e31327c` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:04.302829 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5688a81e-98b2-4e8b-a6fa-d71e3fb327e4 None admin] RBAC: Authorizing `identity:get_user(user_id=65cde5484e14423082cc24d75e31327c)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:04.304352 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5688a81e-98b2-4e8b-a6fa-d71e3fb327e4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:04.306220 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 575/1143] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:56:04 2022] GET /identity/v3/users/65cde5484e14423082cc24d75e31327c => generated 286 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:04.312879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d675dd-72f2-4b1f-94ae-453a3eb83f68 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:04.326387 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:04.436473 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d675dd-72f2-4b1f-94ae-453a3eb83f68 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:04.479525 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cc3bd-2273-45a5-982e-3d2aa64ba571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:04.480166 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cc3bd-2273-45a5-982e-3d2aa64ba571 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:04.480464 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cc3bd-2273-45a5-982e-3d2aa64ba571 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:04.480730 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cc3bd-2273-45a5-982e-3d2aa64ba571 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:04.582325 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19cc3bd-2273-45a5-982e-3d2aa64ba571 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:04.584336 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19cc3bd-2273-45a5-982e-3d2aa64ba571 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:04.597047 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:04.723815 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 569/1144] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:56:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 13:56:04.732215 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5688a81e-98b2-4e8b-a6fa-d71e3fb327e4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:04.861571 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5688a81e-98b2-4e8b-a6fa-d71e3fb327e4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:04.901504 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f4ec8-578f-438b-981e-49903d62de55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:04.902152 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f4ec8-578f-438b-981e-49903d62de55 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:04.902428 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f4ec8-578f-438b-981e-49903d62de55 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:04.902704 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f4ec8-578f-438b-981e-49903d62de55 None admin] PATH_INFO: `/v3/projects/7cf7b338896449848569f0635b4e1efc` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:04.903916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9f4ec8-578f-438b-981e-49903d62de55 None admin] RBAC: Authorizing `identity:get_project(project_id=7cf7b338896449848569f0635b4e1efc)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:04.905435 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9f4ec8-578f-438b-981e-49903d62de55 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:04.907257 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 576/1145] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:56:04 2022] GET /identity/v3/projects/7cf7b338896449848569f0635b4e1efc => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:04.914797 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cc3bd-2273-45a5-982e-3d2aa64ba571 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:05.040953 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cc3bd-2273-45a5-982e-3d2aa64ba571 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:05.054204 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:05.086885 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b36042-1d3f-4d87-80e9-c03f8edac0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:05.087528 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b36042-1d3f-4d87-80e9-c03f8edac0d9 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:05.087806 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b36042-1d3f-4d87-80e9-c03f8edac0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:05.088075 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b36042-1d3f-4d87-80e9-c03f8edac0d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:05.088939 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b36042-1d3f-4d87-80e9-c03f8edac0d9 None admin] RBAC: Adding query filter params (role.id=3e1a9ff016c840c5b51ef7e3da9900ac, scope.project.id=7cf7b338896449848569f0635b4e1efc, user.id=65cde5484e14423082cc24d75e31327c) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:05.089234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b36042-1d3f-4d87-80e9-c03f8edac0d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:05.090709 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b36042-1d3f-4d87-80e9-c03f8edac0d9 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:05.097295 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 570/1146] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:56:04 2022] GET /identity/v3/role_assignments?user.id=65cde5484e14423082cc24d75e31327c&scope.project.id=7cf7b338896449848569f0635b4e1efc&role.id=3e1a9ff016c840c5b51ef7e3da9900ac => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:07.182203 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc9eca-827a-40ad-93f5-fe5e9d700414 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:07.182781 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc9eca-827a-40ad-93f5-fe5e9d700414 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:07.183077 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc9eca-827a-40ad-93f5-fe5e9d700414 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:07.185511 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 577/1147] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:56:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:07.193858 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8418f-032e-4593-a029-189df0262700 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:07.194606 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8418f-032e-4593-a029-189df0262700 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:07.194606 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8418f-032e-4593-a029-189df0262700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:07.215007 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-66e8418f-032e-4593-a029-189df0262700 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:07.216345 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-66e8418f-032e-4593-a029-189df0262700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:07.298939 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 571/1148] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:56:07.324501 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc9eca-827a-40ad-93f5-fe5e9d700414 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:07.347282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:07.453084 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc9eca-827a-40ad-93f5-fe5e9d700414 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:07.503857 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8345877b-7247-4cef-8762-cc77c16566e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:07.504865 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345877b-7247-4cef-8762-cc77c16566e2 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:07.505319 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345877b-7247-4cef-8762-cc77c16566e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:07.505764 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345877b-7247-4cef-8762-cc77c16566e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:07.522416 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8345877b-7247-4cef-8762-cc77c16566e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:07.528032 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-8345877b-7247-4cef-8762-cc77c16566e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:56:07.528032 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:07.528032 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:07.528032 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:07.528032 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:07.528032 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:07.528032 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:07.528032 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:07.528032 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:07.528032 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:07.528032 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:07.528032 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:07.528032 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:07.528032 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:07.528032 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:07.528032 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:07.529964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:07.529964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:07.529964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:07.529964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:07.529964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:07.529964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:07.529964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:07.529964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:07.529964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:07.529964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:07.529964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:07.529964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:07.529964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:07.529964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:07.529964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:07.529964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:07.532400 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:07.532400 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:07.532400 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:07.532400 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:07.532400 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:07.532400 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:07.532400 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:07.532400 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:07.532400 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:07.532400 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:07.532400 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:56:07.532400 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:56:07.532400 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 578/1149] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:56:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:56:07.539964 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e8418f-032e-4593-a029-189df0262700 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:07.693243 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e8418f-032e-4593-a029-189df0262700 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:07.696463 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:07.762187 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76336344-d40c-433f-a32f-f19a596c02d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:07.763273 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76336344-d40c-433f-a32f-f19a596c02d5 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:07.763776 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76336344-d40c-433f-a32f-f19a596c02d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:07.764251 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76336344-d40c-433f-a32f-f19a596c02d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:07.765642 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76336344-d40c-433f-a32f-f19a596c02d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:07.766260 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76336344-d40c-433f-a32f-f19a596c02d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:07.768947 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76336344-d40c-433f-a32f-f19a596c02d5 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:07.785986 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 572/1150] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:56:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:07.795288 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8345877b-7247-4cef-8762-cc77c16566e2 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:07.940419 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:07.973423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8345877b-7247-4cef-8762-cc77c16566e2 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:08.040365 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cdbbb2-5ee2-425d-842b-f6b9f790f7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:08.042535 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cdbbb2-5ee2-425d-842b-f6b9f790f7fb None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:08.043038 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cdbbb2-5ee2-425d-842b-f6b9f790f7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:08.043528 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cdbbb2-5ee2-425d-842b-f6b9f790f7fb None admin] PATH_INFO: `/v3/users/65cde5484e14423082cc24d75e31327c` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:08.045637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cdbbb2-5ee2-425d-842b-f6b9f790f7fb None admin] RBAC: Authorizing `identity:get_user(user_id=65cde5484e14423082cc24d75e31327c)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:08.048400 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cdbbb2-5ee2-425d-842b-f6b9f790f7fb None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:08.051411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 579/1151] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:56:07 2022] GET /identity/v3/users/65cde5484e14423082cc24d75e31327c => generated 286 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:08.061524 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76336344-d40c-433f-a32f-f19a596c02d5 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:08.219135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76336344-d40c-433f-a32f-f19a596c02d5 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:08.265308 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba64cba-8648-4a19-be1b-52e95fcf78b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:08.266446 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba64cba-8648-4a19-be1b-52e95fcf78b0 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:08.266743 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba64cba-8648-4a19-be1b-52e95fcf78b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:08.267017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba64cba-8648-4a19-be1b-52e95fcf78b0 None admin] PATH_INFO: `/v3/projects/7cf7b338896449848569f0635b4e1efc` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:08.268256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba64cba-8648-4a19-be1b-52e95fcf78b0 None admin] RBAC: Authorizing `identity:get_project(project_id=7cf7b338896449848569f0635b4e1efc)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:08.269813 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba64cba-8648-4a19-be1b-52e95fcf78b0 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:08.271617 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 573/1152] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:56:08 2022] GET /identity/v3/projects/7cf7b338896449848569f0635b4e1efc => generated 311 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:08.279181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cdbbb2-5ee2-425d-842b-f6b9f790f7fb None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:08.399397 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cdbbb2-5ee2-425d-842b-f6b9f790f7fb None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:08.445697 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc91f8-bfdf-4aa4-8d87-703b85dd170d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:08.446491 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc91f8-bfdf-4aa4-8d87-703b85dd170d None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:08.446826 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc91f8-bfdf-4aa4-8d87-703b85dd170d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:08.447149 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc91f8-bfdf-4aa4-8d87-703b85dd170d None admin] PATH_INFO: `/v3/projects/7cf7b338896449848569f0635b4e1efc/users/65cde5484e14423082cc24d75e31327c/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:08.449251 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc91f8-bfdf-4aa4-8d87-703b85dd170d None admin] RBAC: Authorizing `identity:create_grant(project_id=7cf7b338896449848569f0635b4e1efc, user_id=65cde5484e14423082cc24d75e31327c, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:08.450760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc91f8-bfdf-4aa4-8d87-703b85dd170d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:08.468197 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 580/1153] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 13:56:08 2022] PUT /identity/v3/projects/7cf7b338896449848569f0635b4e1efc/users/65cde5484e14423082cc24d75e31327c/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:56:10.943878 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729edcda-5c99-4464-9b31-7a6737972c2a None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:10.944630 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729edcda-5c99-4464-9b31-7a6737972c2a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:10.944765 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729edcda-5c99-4464-9b31-7a6737972c2a None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:10.947402 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 574/1154] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:56:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:10.957266 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3c57d-a38b-46e0-ab30-3ba347f08c90 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:10.957840 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3c57d-a38b-46e0-ab30-3ba347f08c90 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:10.958247 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3c57d-a38b-46e0-ab30-3ba347f08c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:10.981243 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:10.990310 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-9da3c57d-a38b-46e0-ab30-3ba347f08c90 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:10.992374 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9da3c57d-a38b-46e0-ab30-3ba347f08c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:11.121919 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 581/1155] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:56:11.146395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729edcda-5c99-4464-9b31-7a6737972c2a None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:11.203197 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:11.311804 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729edcda-5c99-4464-9b31-7a6737972c2a None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:11.361779 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bed643-7e9b-4429-be14-4a9e1dfb5c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:11.362438 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bed643-7e9b-4429-be14-4a9e1dfb5c10 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:11.362719 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bed643-7e9b-4429-be14-4a9e1dfb5c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:11.362987 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bed643-7e9b-4429-be14-4a9e1dfb5c10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:11.382993 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bed643-7e9b-4429-be14-4a9e1dfb5c10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:11.388220 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-a8bed643-7e9b-4429-be14-4a9e1dfb5c10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:56:11.388220 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:11.388220 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:11.388220 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:11.388220 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:11.388220 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:11.388220 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:11.388220 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:11.388220 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:11.388220 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:11.388220 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:11.388220 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:11.388220 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:11.388220 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:11.388220 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:11.388220 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:11.396489 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:11.396489 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:11.396489 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:11.396489 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:11.396489 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:11.396489 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:11.396489 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:11.396489 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:11.396489 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:11.396489 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:11.396489 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:11.396489 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:11.396489 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:11.396489 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:11.396489 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:11.396489 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:11.398441 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:11.398441 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:11.398441 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:11.398441 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:11.398441 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:11.398441 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:11.398441 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:11.398441 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:11.398441 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:11.398441 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:11.398441 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:56:11.398441 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:56:11.398441 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 575/1156] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:56:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:11.400507 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3c57d-a38b-46e0-ab30-3ba347f08c90 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:11.538107 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3c57d-a38b-46e0-ab30-3ba347f08c90 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:11.583148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5c8b6-7461-46a6-af67-a01b708d4e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:11.583774 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5c8b6-7461-46a6-af67-a01b708d4e50 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:11.584065 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5c8b6-7461-46a6-af67-a01b708d4e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:11.584362 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5c8b6-7461-46a6-af67-a01b708d4e50 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:11.585103 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf5c8b6-7461-46a6-af67-a01b708d4e50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:11.585396 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf5c8b6-7461-46a6-af67-a01b708d4e50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:11.586784 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf5c8b6-7461-46a6-af67-a01b708d4e50 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:11.589587 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:11.599197 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 582/1157] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:56:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:11.605269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bed643-7e9b-4429-be14-4a9e1dfb5c10 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:11.719057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:11.730464 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bed643-7e9b-4429-be14-4a9e1dfb5c10 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:11.776449 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8a92b-316e-4f08-a0d0-25f26d395c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:11.777138 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8a92b-316e-4f08-a0d0-25f26d395c8d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:11.777418 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8a92b-316e-4f08-a0d0-25f26d395c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:11.777701 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8a92b-316e-4f08-a0d0-25f26d395c8d None admin] PATH_INFO: `/v3/users/65cde5484e14423082cc24d75e31327c` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:11.778994 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8a92b-316e-4f08-a0d0-25f26d395c8d None admin] RBAC: Authorizing `identity:get_user(user_id=65cde5484e14423082cc24d75e31327c)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:11.780496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8a92b-316e-4f08-a0d0-25f26d395c8d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:11.782336 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 576/1158] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:56:11 2022] GET /identity/v3/users/65cde5484e14423082cc24d75e31327c => generated 286 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:11.789639 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5c8b6-7461-46a6-af67-a01b708d4e50 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:11.918300 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5c8b6-7461-46a6-af67-a01b708d4e50 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:11.968481 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f4216-3152-4be2-8344-2884e13ec3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:11.969889 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f4216-3152-4be2-8344-2884e13ec3cd None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:11.970186 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f4216-3152-4be2-8344-2884e13ec3cd None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:11.970552 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f4216-3152-4be2-8344-2884e13ec3cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:12.063759 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:12.070276 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664f4216-3152-4be2-8344-2884e13ec3cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:12.070736 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664f4216-3152-4be2-8344-2884e13ec3cd None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:12.210562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 583/1159] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:56:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:56:12.218043 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8a92b-316e-4f08-a0d0-25f26d395c8d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:12.360796 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8a92b-316e-4f08-a0d0-25f26d395c8d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:12.412346 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca44ddd-8fd5-4e2d-9292-21ebc6f3302b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:12.413435 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca44ddd-8fd5-4e2d-9292-21ebc6f3302b None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:12.413886 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca44ddd-8fd5-4e2d-9292-21ebc6f3302b None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:12.414358 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca44ddd-8fd5-4e2d-9292-21ebc6f3302b None admin] PATH_INFO: `/v3/projects/7cf7b338896449848569f0635b4e1efc` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:12.416583 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca44ddd-8fd5-4e2d-9292-21ebc6f3302b None admin] RBAC: Authorizing `identity:get_project(project_id=7cf7b338896449848569f0635b4e1efc)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:12.418995 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca44ddd-8fd5-4e2d-9292-21ebc6f3302b None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:12.422227 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 577/1160] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:56:12 2022] GET /identity/v3/projects/7cf7b338896449848569f0635b4e1efc => generated 311 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:12.429333 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f4216-3152-4be2-8344-2884e13ec3cd None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:12.575804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:12.581532 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f4216-3152-4be2-8344-2884e13ec3cd None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:12.635057 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ead60-69fa-4029-b515-1aa4c331f620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:12.635612 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ead60-69fa-4029-b515-1aa4c331f620 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:12.635890 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ead60-69fa-4029-b515-1aa4c331f620 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:12.636166 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ead60-69fa-4029-b515-1aa4c331f620 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:12.637033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015ead60-69fa-4029-b515-1aa4c331f620 None admin] RBAC: Adding query filter params (role.id=3e1a9ff016c840c5b51ef7e3da9900ac, scope.project.id=7cf7b338896449848569f0635b4e1efc, user.id=65cde5484e14423082cc24d75e31327c) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:12.637335 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015ead60-69fa-4029-b515-1aa4c331f620 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:12.638847 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015ead60-69fa-4029-b515-1aa4c331f620 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:12.645849 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 584/1161] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:56:12 2022] GET /identity/v3/role_assignments?user.id=65cde5484e14423082cc24d75e31327c&scope.project.id=7cf7b338896449848569f0635b4e1efc&role.id=3e1a9ff016c840c5b51ef7e3da9900ac => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:14.808196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012f2d7-0e4d-4741-a33b-338872156118 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:14.808823 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012f2d7-0e4d-4741-a33b-338872156118 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:14.808823 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012f2d7-0e4d-4741-a33b-338872156118 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:14.810130 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 578/1162] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:56:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:14.818608 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af13c7-2e37-4e34-b6c1-1c07355e2ee5 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:14.819224 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af13c7-2e37-4e34-b6c1-1c07355e2ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:14.819224 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af13c7-2e37-4e34-b6c1-1c07355e2ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:14.839789 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-78af13c7-2e37-4e34-b6c1-1c07355e2ee5 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:14.841050 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-78af13c7-2e37-4e34-b6c1-1c07355e2ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:14.944992 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 585/1163] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:56:14.969025 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012f2d7-0e4d-4741-a33b-338872156118 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:15.006111 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:15.113955 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012f2d7-0e4d-4741-a33b-338872156118 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:15.163868 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b366504-b641-411f-b516-af1c6861a92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:15.164958 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b366504-b641-411f-b516-af1c6861a92d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:15.165467 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b366504-b641-411f-b516-af1c6861a92d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:15.165952 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b366504-b641-411f-b516-af1c6861a92d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:15.270915 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b366504-b641-411f-b516-af1c6861a92d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:15.272510 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b366504-b641-411f-b516-af1c6861a92d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:15.308176 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:15.422063 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 579/1164] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:56:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:56:15.431465 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af13c7-2e37-4e34-b6c1-1c07355e2ee5 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:15.556945 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:15.572118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af13c7-2e37-4e34-b6c1-1c07355e2ee5 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:15.634366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62da41-ac86-40d4-9cd9-da6f38d1ef73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:15.635045 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62da41-ac86-40d4-9cd9-da6f38d1ef73 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:15.635324 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62da41-ac86-40d4-9cd9-da6f38d1ef73 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:15.635606 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62da41-ac86-40d4-9cd9-da6f38d1ef73 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:15.644183 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62da41-ac86-40d4-9cd9-da6f38d1ef73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:15.645744 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62da41-ac86-40d4-9cd9-da6f38d1ef73 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:15.662686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-8c62da41-ac86-40d4-9cd9-da6f38d1ef73 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 01 13:56:15.662686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:15.662686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 01 13:56:15.662686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 01 13:56:15.662686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:56:15.662686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:56:15.662686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:56:15.662686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:56:15.662686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 01 13:56:15.662686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:56:15.662686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:56:15.662686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:56:15.662686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:15.662686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:15.662686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:15.662686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:15.662686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:15.665160 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:15.665160 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:15.665160 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:15.665160 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:15.665160 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:15.665160 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:15.665160 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:15.665160 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:15.665160 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:15.665160 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:15.665160 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:15.665160 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:15.665160 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:15.665160 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:15.665160 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:15.665160 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:15.666784 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:15.666784 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:15.666784 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:15.666784 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:15.666784 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:15.666784 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:15.666784 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:15.666784 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:15.666784 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:15.666784 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:15.666784 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 01 13:56:15.666784 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:15.666784 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 01 13:56:15.666784 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:56:15.666784 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 586/1165] 10.222.0.45 () {56 vars in 1117 bytes} [Mon Aug 1 13:56:15 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:15.672508 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b366504-b641-411f-b516-af1c6861a92d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:15.799033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b366504-b641-411f-b516-af1c6861a92d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:15.829622 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:15.842695 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d11d5-3b55-4665-abbb-0419e53d7c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:15.843572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d11d5-3b55-4665-abbb-0419e53d7c6b None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:15.843855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d11d5-3b55-4665-abbb-0419e53d7c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:15.844128 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d11d5-3b55-4665-abbb-0419e53d7c6b None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:15.844919 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54d11d5-3b55-4665-abbb-0419e53d7c6b None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:15.845223 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54d11d5-3b55-4665-abbb-0419e53d7c6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:15.846642 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54d11d5-3b55-4665-abbb-0419e53d7c6b None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:15.855708 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 580/1166] 10.222.0.45 () {56 vars in 1115 bytes} [Mon Aug 1 13:56:15 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:17.926415 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd900bf1-f452-4508-9c44-95a7b6e79054 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:17.927166 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd900bf1-f452-4508-9c44-95a7b6e79054 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:17.927884 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd900bf1-f452-4508-9c44-95a7b6e79054 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:17.930178 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 587/1167] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:56:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:17.938655 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1cd98a-d393-46e6-b578-77e8eb0c65c7 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:17.939439 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1cd98a-d393-46e6-b578-77e8eb0c65c7 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:17.939439 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1cd98a-d393-46e6-b578-77e8eb0c65c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:17.959056 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-fa1cd98a-d393-46e6-b578-77e8eb0c65c7 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:17.960344 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-fa1cd98a-d393-46e6-b578-77e8eb0c65c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:18.056890 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 581/1168] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:56:18.081219 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd900bf1-f452-4508-9c44-95a7b6e79054 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:18.225455 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd900bf1-f452-4508-9c44-95a7b6e79054 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:18.262446 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ed26b-dfa0-46d9-8563-eae44f285b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:18.263069 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ed26b-dfa0-46d9-8563-eae44f285b7a None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:18.263373 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ed26b-dfa0-46d9-8563-eae44f285b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:18.263647 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ed26b-dfa0-46d9-8563-eae44f285b7a None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:18.264418 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ed26b-dfa0-46d9-8563-eae44f285b7a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:18.265822 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ed26b-dfa0-46d9-8563-eae44f285b7a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:18.292358 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 588/1169] 10.222.0.45 () {60 vars in 1138 bytes} [Mon Aug 1 13:56:18 2022] POST /identity/v3/domains => generated 267 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 13:56:20.486210 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c003421-d96e-4253-b06c-29231fb427b7 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:20.486818 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c003421-d96e-4253-b06c-29231fb427b7 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:20.486818 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c003421-d96e-4253-b06c-29231fb427b7 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:20.488720 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 582/1170] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:56:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:20.494836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa4cd60-e767-4630-be84-29f284bf9ae7 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:20.495624 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa4cd60-e767-4630-be84-29f284bf9ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:20.495624 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa4cd60-e767-4630-be84-29f284bf9ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:20.519136 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-baa4cd60-e767-4630-be84-29f284bf9ae7 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:20.520421 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-baa4cd60-e767-4630-be84-29f284bf9ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:20.530722 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:20.605957 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 589/1171] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:56:20.630167 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c003421-d96e-4253-b06c-29231fb427b7 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:20.764432 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c003421-d96e-4253-b06c-29231fb427b7 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:20.794281 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:20.810307 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda41ba5-df0a-42cb-9fd0-9ffb8edf37cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:20.812359 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda41ba5-df0a-42cb-9fd0-9ffb8edf37cc None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:20.812359 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda41ba5-df0a-42cb-9fd0-9ffb8edf37cc None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:20.812359 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda41ba5-df0a-42cb-9fd0-9ffb8edf37cc None admin] PATH_INFO: `/v3/domains/657c950bb0c54efb8f2b254a10da5aa5` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:20.832649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda41ba5-df0a-42cb-9fd0-9ffb8edf37cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=657c950bb0c54efb8f2b254a10da5aa5)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:20.835350 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda41ba5-df0a-42cb-9fd0-9ffb8edf37cc None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:20.838777 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 583/1172] 10.222.0.45 () {56 vars in 1183 bytes} [Mon Aug 1 13:56:20 2022] GET /identity/v3/domains/657c950bb0c54efb8f2b254a10da5aa5 => generated 267 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:20.846076 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa4cd60-e767-4630-be84-29f284bf9ae7 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:20.965727 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa4cd60-e767-4630-be84-29f284bf9ae7 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:21.027183 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031ac57-373e-45c8-8876-a9b5c5c78737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:21.028186 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031ac57-373e-45c8-8876-a9b5c5c78737 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:21.028691 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031ac57-373e-45c8-8876-a9b5c5c78737 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:21.029147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031ac57-373e-45c8-8876-a9b5c5c78737 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:21.030407 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0031ac57-373e-45c8-8876-a9b5c5c78737 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:21.033468 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0031ac57-373e-45c8-8876-a9b5c5c78737 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:21.055724 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:21.077447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 590/1173] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Aug 1 13:56:20 2022] POST /identity/v3/projects => generated 361 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:56:23.415685 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b5a39-f2f3-446a-b97f-e544da74c63a None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:23.416566 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b5a39-f2f3-446a-b97f-e544da74c63a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:23.416566 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b5a39-f2f3-446a-b97f-e544da74c63a None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:23.417829 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 584/1174] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:56:23.424817 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ce875-1322-4793-ae21-ceb558ea5926 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:23.425442 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ce875-1322-4793-ae21-ceb558ea5926 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:23.425780 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ce875-1322-4793-ae21-ceb558ea5926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:23.458951 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-7f0ce875-1322-4793-ae21-ceb558ea5926 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:23.461247 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7f0ce875-1322-4793-ae21-ceb558ea5926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:23.553629 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 591/1175] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:56:23.578320 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b5a39-f2f3-446a-b97f-e544da74c63a None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:23.703693 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b5a39-f2f3-446a-b97f-e544da74c63a None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:23.747140 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efad18d-6b64-4ebd-ab6b-7140e918e7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:23.747816 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efad18d-6b64-4ebd-ab6b-7140e918e7db None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:23.748166 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efad18d-6b64-4ebd-ab6b-7140e918e7db None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:23.748567 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efad18d-6b64-4ebd-ab6b-7140e918e7db None admin] PATH_INFO: `/v3/domains/657c950bb0c54efb8f2b254a10da5aa5` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:23.750683 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efad18d-6b64-4ebd-ab6b-7140e918e7db None admin] RBAC: Authorizing `identity:get_domain(domain_id=657c950bb0c54efb8f2b254a10da5aa5)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:23.753480 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efad18d-6b64-4ebd-ab6b-7140e918e7db None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:23.756350 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 585/1176] 10.222.0.45 () {56 vars in 1183 bytes} [Mon Aug 1 13:56:23 2022] GET /identity/v3/domains/657c950bb0c54efb8f2b254a10da5aa5 => generated 267 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:56:23.762710 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ce875-1322-4793-ae21-ceb558ea5926 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:23.927951 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ce875-1322-4793-ae21-ceb558ea5926 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:23.967360 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:23.980955 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c9f63-6742-4ba3-8a43-a235879747ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:23.981639 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c9f63-6742-4ba3-8a43-a235879747ea None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:23.981957 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c9f63-6742-4ba3-8a43-a235879747ea None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:23.982287 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c9f63-6742-4ba3-8a43-a235879747ea None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:23.983169 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625c9f63-6742-4ba3-8a43-a235879747ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:23.984555 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625c9f63-6742-4ba3-8a43-a235879747ea None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:24.009320 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 592/1177] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 13:56:23 2022] POST /identity/v3/users => generated 311 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 13:56:26.186056 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7460a-4fff-4686-8d47-f2d4906a6519 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:26.186669 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7460a-4fff-4686-8d47-f2d4906a6519 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:26.186993 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7460a-4fff-4686-8d47-f2d4906a6519 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:26.189313 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 586/1178] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:56:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:26.198362 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966622f-6bb2-4fdf-a19c-e85e208f1f5b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:26.198958 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966622f-6bb2-4fdf-a19c-e85e208f1f5b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:26.199255 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966622f-6bb2-4fdf-a19c-e85e208f1f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:26.222864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-4966622f-6bb2-4fdf-a19c-e85e208f1f5b None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:26.224759 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4966622f-6bb2-4fdf-a19c-e85e208f1f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:26.312070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 593/1179] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:56:26.335804 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7460a-4fff-4686-8d47-f2d4906a6519 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:26.402377 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:26.469271 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7460a-4fff-4686-8d47-f2d4906a6519 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:26.510716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7fba69-fbd2-4911-89b9-b397ca43171d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:26.511373 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fba69-fbd2-4911-89b9-b397ca43171d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:26.511649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fba69-fbd2-4911-89b9-b397ca43171d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:26.511918 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fba69-fbd2-4911-89b9-b397ca43171d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:26.525691 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7fba69-fbd2-4911-89b9-b397ca43171d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:26.530641 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-bd7fba69-fbd2-4911-89b9-b397ca43171d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:56:26.530641 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:26.530641 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:26.530641 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:26.530641 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:26.530641 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:26.530641 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:26.530641 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:26.530641 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:26.530641 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:26.530641 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:26.530641 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:26.530641 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:26.530641 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:26.530641 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:26.530641 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:26.531776 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:26.531776 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:26.531776 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:26.531776 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:26.531776 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:26.531776 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:26.531776 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:26.531776 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:26.531776 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:26.531776 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:26.531776 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:26.531776 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:26.531776 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:26.531776 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:26.531776 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:26.531776 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:26.532833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:26.532833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:26.532833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:26.532833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:26.532833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:26.532833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:26.532833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:26.532833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:26.532833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:26.532833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:26.532833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:56:26.532833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:56:26.536501 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 587/1180] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:56:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:56:26.538459 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966622f-6bb2-4fdf-a19c-e85e208f1f5b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:26.670719 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966622f-6bb2-4fdf-a19c-e85e208f1f5b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:26.687833 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:26.718409 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd78644-210f-4eeb-88b4-1d35d0e6df45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:26.719042 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd78644-210f-4eeb-88b4-1d35d0e6df45 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:26.719402 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd78644-210f-4eeb-88b4-1d35d0e6df45 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:26.719645 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd78644-210f-4eeb-88b4-1d35d0e6df45 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:26.720408 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd78644-210f-4eeb-88b4-1d35d0e6df45 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:26.720707 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd78644-210f-4eeb-88b4-1d35d0e6df45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:26.722138 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd78644-210f-4eeb-88b4-1d35d0e6df45 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:26.731929 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 594/1181] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:56:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:56:26.738580 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7fba69-fbd2-4911-89b9-b397ca43171d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:26.872870 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7fba69-fbd2-4911-89b9-b397ca43171d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:26.885535 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:26.913345 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc6c1c-515e-41cd-9710-8b10072dc79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:26.914016 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc6c1c-515e-41cd-9710-8b10072dc79e None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:26.914408 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc6c1c-515e-41cd-9710-8b10072dc79e None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:26.914885 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc6c1c-515e-41cd-9710-8b10072dc79e None admin] PATH_INFO: `/v3/users/8ab395a23fd14ad58235234fdfd9e23a` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:26.930910 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc6c1c-515e-41cd-9710-8b10072dc79e None admin] RBAC: Authorizing `identity:get_user(user_id=8ab395a23fd14ad58235234fdfd9e23a)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:26.932414 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc6c1c-515e-41cd-9710-8b10072dc79e None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:26.934167 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 588/1182] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:56:26 2022] GET /identity/v3/users/8ab395a23fd14ad58235234fdfd9e23a => generated 311 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:26.940427 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd78644-210f-4eeb-88b4-1d35d0e6df45 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:27.058411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd78644-210f-4eeb-88b4-1d35d0e6df45 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:27.102439 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172645af-0a94-4487-a5d5-763ba9854055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:27.103153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172645af-0a94-4487-a5d5-763ba9854055 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:27.103465 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172645af-0a94-4487-a5d5-763ba9854055 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:27.103809 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172645af-0a94-4487-a5d5-763ba9854055 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:27.156923 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:27.196258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172645af-0a94-4487-a5d5-763ba9854055 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:27.198585 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172645af-0a94-4487-a5d5-763ba9854055 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:27.325495 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 595/1183] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:56:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:56:27.334770 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc6c1c-515e-41cd-9710-8b10072dc79e None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:27.458557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc6c1c-515e-41cd-9710-8b10072dc79e None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:27.500456 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3886e-04de-44e2-8139-88da43eb9d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:27.501277 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3886e-04de-44e2-8139-88da43eb9d6d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:27.501701 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3886e-04de-44e2-8139-88da43eb9d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:27.502019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3886e-04de-44e2-8139-88da43eb9d6d None admin] PATH_INFO: `/v3/projects/1b845d2884df47048734310834953ead` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:27.503322 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e3886e-04de-44e2-8139-88da43eb9d6d None admin] RBAC: Authorizing `identity:get_project(project_id=1b845d2884df47048734310834953ead)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:27.505153 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e3886e-04de-44e2-8139-88da43eb9d6d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:27.507071 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 589/1184] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:56:27 2022] GET /identity/v3/projects/1b845d2884df47048734310834953ead => generated 361 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:27.513823 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172645af-0a94-4487-a5d5-763ba9854055 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:27.612400 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:27.658681 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172645af-0a94-4487-a5d5-763ba9854055 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:27.702189 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe783a17-f1a4-431b-ba44-7b5d1ea5214b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:27.703210 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe783a17-f1a4-431b-ba44-7b5d1ea5214b None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:27.703624 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe783a17-f1a4-431b-ba44-7b5d1ea5214b None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:27.704129 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe783a17-f1a4-431b-ba44-7b5d1ea5214b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:27.704940 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe783a17-f1a4-431b-ba44-7b5d1ea5214b None admin] RBAC: Adding query filter params (role.id=3e1a9ff016c840c5b51ef7e3da9900ac, scope.project.id=1b845d2884df47048734310834953ead, user.id=8ab395a23fd14ad58235234fdfd9e23a) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:27.705250 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe783a17-f1a4-431b-ba44-7b5d1ea5214b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:27.706720 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe783a17-f1a4-431b-ba44-7b5d1ea5214b None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:27.713347 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 596/1185] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:56:27 2022] GET /identity/v3/role_assignments?user.id=8ab395a23fd14ad58235234fdfd9e23a&scope.project.id=1b845d2884df47048734310834953ead&role.id=3e1a9ff016c840c5b51ef7e3da9900ac => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:29.761363 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d309e7-9d57-4d3a-b085-26afc1cd6f43 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:29.761908 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d309e7-9d57-4d3a-b085-26afc1cd6f43 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:29.761908 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d309e7-9d57-4d3a-b085-26afc1cd6f43 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:29.763315 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 590/1186] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:56:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:29.769320 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e62a51-bf4f-4b26-a1d0-60d93b9878f6 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:29.769821 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e62a51-bf4f-4b26-a1d0-60d93b9878f6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:29.769932 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e62a51-bf4f-4b26-a1d0-60d93b9878f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:29.789908 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-69e62a51-bf4f-4b26-a1d0-60d93b9878f6 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:29.791166 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-69e62a51-bf4f-4b26-a1d0-60d93b9878f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:29.866498 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 597/1187] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:56:29.889426 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d309e7-9d57-4d3a-b085-26afc1cd6f43 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:29.932221 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:30.017385 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d309e7-9d57-4d3a-b085-26afc1cd6f43 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:30.055455 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad189e0b-ae16-404e-822b-3445f563e485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:30.056108 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad189e0b-ae16-404e-822b-3445f563e485 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:30.056418 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad189e0b-ae16-404e-822b-3445f563e485 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:30.056695 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad189e0b-ae16-404e-822b-3445f563e485 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:30.068732 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad189e0b-ae16-404e-822b-3445f563e485 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:30.073825 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-ad189e0b-ae16-404e-822b-3445f563e485 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:56:30.073825 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:30.073825 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:30.073825 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:30.073825 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:30.073825 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:30.073825 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:30.073825 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:30.073825 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:30.073825 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:30.073825 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:30.073825 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:30.073825 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:30.073825 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:30.073825 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:30.073825 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:30.075614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:30.075614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:30.075614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:30.075614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:30.075614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:30.075614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:30.075614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:30.075614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:30.075614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:30.075614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:30.075614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:30.075614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:30.075614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:30.075614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:30.075614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:30.075614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:30.077399 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:30.077399 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:30.077399 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:30.077399 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:30.077399 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:30.077399 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:30.077399 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:30.077399 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:30.077399 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:30.077399 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:30.077399 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:56:30.077399 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:56:30.077399 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 591/1188] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:56:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:30.082313 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e62a51-bf4f-4b26-a1d0-60d93b9878f6 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:30.151008 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:30.187259 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e62a51-bf4f-4b26-a1d0-60d93b9878f6 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:30.222293 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a3d4c-0335-4b1b-966b-3c6450b82df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:30.222923 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a3d4c-0335-4b1b-966b-3c6450b82df0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:30.223204 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a3d4c-0335-4b1b-966b-3c6450b82df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:30.223483 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a3d4c-0335-4b1b-966b-3c6450b82df0 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:30.224209 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690a3d4c-0335-4b1b-966b-3c6450b82df0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:30.224548 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690a3d4c-0335-4b1b-966b-3c6450b82df0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:30.225928 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690a3d4c-0335-4b1b-966b-3c6450b82df0 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:30.234726 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 598/1189] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:56:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:30.243256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad189e0b-ae16-404e-822b-3445f563e485 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:30.351275 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:30.358074 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad189e0b-ae16-404e-822b-3445f563e485 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:30.397778 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835fed77-e28e-42c7-864c-6a36b3c46186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:30.398430 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835fed77-e28e-42c7-864c-6a36b3c46186 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:30.399145 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835fed77-e28e-42c7-864c-6a36b3c46186 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:30.399350 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835fed77-e28e-42c7-864c-6a36b3c46186 None admin] PATH_INFO: `/v3/users/8ab395a23fd14ad58235234fdfd9e23a` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:30.400619 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835fed77-e28e-42c7-864c-6a36b3c46186 None admin] RBAC: Authorizing `identity:get_user(user_id=8ab395a23fd14ad58235234fdfd9e23a)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:30.402150 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835fed77-e28e-42c7-864c-6a36b3c46186 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:30.403789 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 592/1190] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:56:30 2022] GET /identity/v3/users/8ab395a23fd14ad58235234fdfd9e23a => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:30.410026 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a3d4c-0335-4b1b-966b-3c6450b82df0 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:30.529368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a3d4c-0335-4b1b-966b-3c6450b82df0 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:30.560740 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:30.570074 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69047f1-884f-4778-b017-f85d636b1a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:30.570733 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69047f1-884f-4778-b017-f85d636b1a20 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:30.571014 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69047f1-884f-4778-b017-f85d636b1a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:30.571286 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69047f1-884f-4778-b017-f85d636b1a20 None admin] PATH_INFO: `/v3/projects/1b845d2884df47048734310834953ead` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:30.572516 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69047f1-884f-4778-b017-f85d636b1a20 None admin] RBAC: Authorizing `identity:get_project(project_id=1b845d2884df47048734310834953ead)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:30.573983 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69047f1-884f-4778-b017-f85d636b1a20 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:30.575734 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 599/1191] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:56:30 2022] GET /identity/v3/projects/1b845d2884df47048734310834953ead => generated 361 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:56:30.583064 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835fed77-e28e-42c7-864c-6a36b3c46186 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:30.701474 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835fed77-e28e-42c7-864c-6a36b3c46186 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:30.745442 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe82fd-7bb4-417a-bc6e-8addaa7a136c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:30.746125 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe82fd-7bb4-417a-bc6e-8addaa7a136c None admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:30.746396 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe82fd-7bb4-417a-bc6e-8addaa7a136c None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:30.746672 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe82fd-7bb4-417a-bc6e-8addaa7a136c None admin] PATH_INFO: `/v3/projects/1b845d2884df47048734310834953ead/users/8ab395a23fd14ad58235234fdfd9e23a/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:30.749003 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe82fd-7bb4-417a-bc6e-8addaa7a136c None admin] RBAC: Authorizing `identity:create_grant(project_id=1b845d2884df47048734310834953ead, user_id=8ab395a23fd14ad58235234fdfd9e23a, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:30.750478 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe82fd-7bb4-417a-bc6e-8addaa7a136c None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:30.765667 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 593/1192] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 13:56:30 2022] PUT /identity/v3/projects/1b845d2884df47048734310834953ead/users/8ab395a23fd14ad58235234fdfd9e23a/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:56:32.850712 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d45797-4a5c-49cc-9d32-0d3cc82b1da6 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:32.851323 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d45797-4a5c-49cc-9d32-0d3cc82b1da6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:32.851476 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d45797-4a5c-49cc-9d32-0d3cc82b1da6 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:32.853720 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 600/1193] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:56:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:32.861984 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf41f0-4bf9-4b53-a883-7c8210f0327b None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:32.862818 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf41f0-4bf9-4b53-a883-7c8210f0327b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:32.862818 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf41f0-4bf9-4b53-a883-7c8210f0327b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:32.883453 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-35bf41f0-4bf9-4b53-a883-7c8210f0327b None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:32.884840 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-35bf41f0-4bf9-4b53-a883-7c8210f0327b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:32.906891 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:32.985173 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 594/1194] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:56:33.009983 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d45797-4a5c-49cc-9d32-0d3cc82b1da6 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:33.189807 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d45797-4a5c-49cc-9d32-0d3cc82b1da6 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:33.251209 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d386a-cf58-40f7-9937-c0ac3a61c436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:33.252300 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d386a-cf58-40f7-9937-c0ac3a61c436 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:33.252757 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d386a-cf58-40f7-9937-c0ac3a61c436 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:33.253186 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d386a-cf58-40f7-9937-c0ac3a61c436 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:33.270902 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d386a-cf58-40f7-9937-c0ac3a61c436 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:33.276385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-936d386a-cf58-40f7-9937-c0ac3a61c436 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:56:33.276385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:33.276385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:33.276385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:33.276385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:33.276385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:33.276385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:33.276385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:33.276385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:33.276385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:33.276385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:33.276385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:33.276385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:33.276385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:33.276385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:33.276385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:33.277562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:33.277562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:33.277562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:33.277562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:33.277562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:33.277562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:33.277562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:33.277562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:33.277562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:33.277562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:33.277562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:33.277562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:33.277562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:33.277562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:33.277562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:33.277562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:33.278585 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:33.278585 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:33.278585 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:33.278585 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:33.278585 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:33.278585 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:33.278585 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:33.278585 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:33.278585 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:33.278585 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:33.278585 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:56:33.278585 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:56:33.278585 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 601/1195] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:56:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:33.287622 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf41f0-4bf9-4b53-a883-7c8210f0327b None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:33.425654 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf41f0-4bf9-4b53-a883-7c8210f0327b None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:33.467890 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067527b3-62d4-4c1b-8ae6-38f11422d990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:33.468824 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067527b3-62d4-4c1b-8ae6-38f11422d990 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:33.469269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067527b3-62d4-4c1b-8ae6-38f11422d990 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:33.469715 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067527b3-62d4-4c1b-8ae6-38f11422d990 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:33.474461 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067527b3-62d4-4c1b-8ae6-38f11422d990 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:33.474959 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067527b3-62d4-4c1b-8ae6-38f11422d990 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:33.476852 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067527b3-62d4-4c1b-8ae6-38f11422d990 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:33.488212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 595/1196] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:56:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:33.497463 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d386a-cf58-40f7-9937-c0ac3a61c436 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:33.568489 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:33.624011 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d386a-cf58-40f7-9937-c0ac3a61c436 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:33.665688 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c73fc3-14b1-4f67-b780-42428f8a6e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:33.666343 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c73fc3-14b1-4f67-b780-42428f8a6e55 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:33.666622 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c73fc3-14b1-4f67-b780-42428f8a6e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:33.667022 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c73fc3-14b1-4f67-b780-42428f8a6e55 None admin] PATH_INFO: `/v3/users/8ab395a23fd14ad58235234fdfd9e23a` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:33.668410 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c73fc3-14b1-4f67-b780-42428f8a6e55 None admin] RBAC: Authorizing `identity:get_user(user_id=8ab395a23fd14ad58235234fdfd9e23a)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:33.669910 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c73fc3-14b1-4f67-b780-42428f8a6e55 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:33.671594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 602/1197] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 13:56:33 2022] GET /identity/v3/users/8ab395a23fd14ad58235234fdfd9e23a => generated 311 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:33.678257 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067527b3-62d4-4c1b-8ae6-38f11422d990 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:33.706908 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:33.796734 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067527b3-62d4-4c1b-8ae6-38f11422d990 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:33.832023 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbbad12-6fc1-4d3b-a4f9-9429101af036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:33.832672 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbbad12-6fc1-4d3b-a4f9-9429101af036 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:33.832947 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbbad12-6fc1-4d3b-a4f9-9429101af036 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:33.833215 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbbad12-6fc1-4d3b-a4f9-9429101af036 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:33.903531 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbbad12-6fc1-4d3b-a4f9-9429101af036 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:33.905186 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbbad12-6fc1-4d3b-a4f9-9429101af036 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:33.922078 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:34.005100 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 596/1198] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:56:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:56:34.011250 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c73fc3-14b1-4f67-b780-42428f8a6e55 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:34.118695 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c73fc3-14b1-4f67-b780-42428f8a6e55 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:34.153526 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:34.166255 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9695654-0003-4ff4-91de-5dc2244dbefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:34.167207 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9695654-0003-4ff4-91de-5dc2244dbefa None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:34.167492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9695654-0003-4ff4-91de-5dc2244dbefa None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:34.167764 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9695654-0003-4ff4-91de-5dc2244dbefa None admin] PATH_INFO: `/v3/projects/1b845d2884df47048734310834953ead` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:34.168956 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9695654-0003-4ff4-91de-5dc2244dbefa None admin] RBAC: Authorizing `identity:get_project(project_id=1b845d2884df47048734310834953ead)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:34.170400 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9695654-0003-4ff4-91de-5dc2244dbefa None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:34.172105 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 603/1199] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 13:56:34 2022] GET /identity/v3/projects/1b845d2884df47048734310834953ead => generated 361 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:34.178776 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbbad12-6fc1-4d3b-a4f9-9429101af036 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:34.293374 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbbad12-6fc1-4d3b-a4f9-9429101af036 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:34.314606 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:34.339852 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199cb84a-925e-4c5b-a354-fe2455ffe7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:34.340506 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199cb84a-925e-4c5b-a354-fe2455ffe7f5 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:34.340782 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199cb84a-925e-4c5b-a354-fe2455ffe7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:34.341049 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199cb84a-925e-4c5b-a354-fe2455ffe7f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:34.341845 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199cb84a-925e-4c5b-a354-fe2455ffe7f5 None admin] RBAC: Adding query filter params (role.id=3e1a9ff016c840c5b51ef7e3da9900ac, scope.project.id=1b845d2884df47048734310834953ead, user.id=8ab395a23fd14ad58235234fdfd9e23a) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:34.342150 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199cb84a-925e-4c5b-a354-fe2455ffe7f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:34.343595 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199cb84a-925e-4c5b-a354-fe2455ffe7f5 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:34.349968 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 597/1200] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:56:34 2022] GET /identity/v3/role_assignments?user.id=8ab395a23fd14ad58235234fdfd9e23a&scope.project.id=1b845d2884df47048734310834953ead&role.id=3e1a9ff016c840c5b51ef7e3da9900ac => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:44.160633 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:44.347268 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=10 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:32.306775 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1992506-5098-4493-9e80-b13c7854af5b None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:32.307730 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1992506-5098-4493-9e80-b13c7854af5b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:32.307730 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1992506-5098-4493-9e80-b13c7854af5b None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:32.309314 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 604/1201] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:57:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:57:32.315681 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e0fff-22cd-4bab-a385-5090ea0955b5 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:32.316535 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e0fff-22cd-4bab-a385-5090ea0955b5 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:32.316535 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e0fff-22cd-4bab-a385-5090ea0955b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:32.348977 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-5e0e0fff-22cd-4bab-a385-5090ea0955b5 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:32.350972 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5e0e0fff-22cd-4bab-a385-5090ea0955b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:32.474802 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 598/1202] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:57:32.504908 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1992506-5098-4493-9e80-b13c7854af5b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:32.699069 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1992506-5098-4493-9e80-b13c7854af5b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:32.759920 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1435371e-9bc4-4808-99bf-8de165bc7caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:32.760739 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435371e-9bc4-4808-99bf-8de165bc7caf None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:32.761153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435371e-9bc4-4808-99bf-8de165bc7caf None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:32.761304 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435371e-9bc4-4808-99bf-8de165bc7caf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:32.786347 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1435371e-9bc4-4808-99bf-8de165bc7caf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:32.788147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1435371e-9bc4-4808-99bf-8de165bc7caf None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:32.816063 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-1435371e-9bc4-4808-99bf-8de165bc7caf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:32.816063 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:32.816063 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:32.816063 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:32.816063 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:32.816063 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:32.816063 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:32.816063 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:32.816063 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:32.816063 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:32.816063 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:32.816063 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:32.816063 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:57:32.816063 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:57:32.816063 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:57:32.818282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:57:32.818282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:32.818282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:32.818282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:32.818282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:32.818282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:32.818282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:32.818282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:32.818282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:32.818282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:32.818282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:32.818282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:32.818282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:32.818282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:32.818282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:32.818282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:32.818282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:32.820212 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:57:32.820212 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:57:32.820212 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:32.820212 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:57:32.820212 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 605/1203] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:57:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:32.826907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e0fff-22cd-4bab-a385-5090ea0955b5 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:32.993519 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e0fff-22cd-4bab-a385-5090ea0955b5 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:33.051739 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:33.067579 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192dbe08-adfe-4356-b388-20ab26cb8d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:33.068265 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192dbe08-adfe-4356-b388-20ab26cb8d13 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:33.068626 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192dbe08-adfe-4356-b388-20ab26cb8d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:33.068899 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192dbe08-adfe-4356-b388-20ab26cb8d13 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:33.069616 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192dbe08-adfe-4356-b388-20ab26cb8d13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:33.069932 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192dbe08-adfe-4356-b388-20ab26cb8d13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:33.071416 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192dbe08-adfe-4356-b388-20ab26cb8d13 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:33.085901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 599/1204] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:57:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:33.092753 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1435371e-9bc4-4808-99bf-8de165bc7caf None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:33.156678 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:33.223389 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1435371e-9bc4-4808-99bf-8de165bc7caf None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:33.283630 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9a6fb-30de-4692-bc8c-481fdf79fcf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:33.284474 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9a6fb-30de-4692-bc8c-481fdf79fcf3 None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:33.284825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9a6fb-30de-4692-bc8c-481fdf79fcf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:33.285182 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9a6fb-30de-4692-bc8c-481fdf79fcf3 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:33.286230 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9a6fb-30de-4692-bc8c-481fdf79fcf3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:33.287703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9a6fb-30de-4692-bc8c-481fdf79fcf3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:33.328323 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 606/1205] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 13:57:33 2022] POST /identity/v3/users => generated 251 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 13:57:35.775429 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b3b11-910d-4d99-bdc8-a2e5307323e8 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:35.775994 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b3b11-910d-4d99-bdc8-a2e5307323e8 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:35.776862 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b3b11-910d-4d99-bdc8-a2e5307323e8 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:35.780186 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 600/1206] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:57:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:57:35.787198 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3ecb3-0e87-4600-a969-a78a33a7f496 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:35.787802 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3ecb3-0e87-4600-a969-a78a33a7f496 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:35.788054 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3ecb3-0e87-4600-a969-a78a33a7f496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:35.822000 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c2a3ecb3-0e87-4600-a969-a78a33a7f496 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:35.823680 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c2a3ecb3-0e87-4600-a969-a78a33a7f496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:35.923376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 607/1207] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:57:35.946199 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b3b11-910d-4d99-bdc8-a2e5307323e8 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:36.065316 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b3b11-910d-4d99-bdc8-a2e5307323e8 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:36.103884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae308c3-68ce-4076-9343-0f3d335528c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:36.104667 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae308c3-68ce-4076-9343-0f3d335528c7 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:36.104958 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae308c3-68ce-4076-9343-0f3d335528c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:36.105234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae308c3-68ce-4076-9343-0f3d335528c7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:36.118820 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae308c3-68ce-4076-9343-0f3d335528c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:57:36.123960 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-5ae308c3-68ce-4076-9343-0f3d335528c7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:57:36.123960 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:36.123960 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:36.123960 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:36.123960 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:36.123960 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:36.123960 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:36.123960 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:36.123960 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:36.123960 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:36.123960 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:36.123960 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:36.123960 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:57:36.123960 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:57:36.123960 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:57:36.123960 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:57:36.125299 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:57:36.125299 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:57:36.125299 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:57:36.125299 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:57:36.125299 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:36.125299 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:36.125299 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:36.125299 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:36.125299 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:36.125299 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:36.125299 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:36.125299 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:36.125299 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:36.125299 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:36.125299 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:36.125299 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:36.126313 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:36.126313 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:36.126313 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:36.126313 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:36.126313 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:57:36.126313 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:57:36.126313 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:57:36.126313 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:57:36.126313 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:57:36.126313 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:57:36.126313 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:57:36.126313 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:57:36.126313 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 601/1208] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 13:57:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:36.132290 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a3ecb3-0e87-4600-a969-a78a33a7f496 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:36.184415 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:36.270549 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a3ecb3-0e87-4600-a969-a78a33a7f496 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:36.319777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7bd6c-0910-4b7f-8153-aa7433deeae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:36.320521 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7bd6c-0910-4b7f-8153-aa7433deeae4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:36.320804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7bd6c-0910-4b7f-8153-aa7433deeae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:36.321126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7bd6c-0910-4b7f-8153-aa7433deeae4 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:36.321867 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa7bd6c-0910-4b7f-8153-aa7433deeae4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:36.322172 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa7bd6c-0910-4b7f-8153-aa7433deeae4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:36.323561 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa7bd6c-0910-4b7f-8153-aa7433deeae4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:36.335880 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 608/1209] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 13:57:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:36.341780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae308c3-68ce-4076-9343-0f3d335528c7 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:36.419433 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:36.498155 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae308c3-68ce-4076-9343-0f3d335528c7 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:36.542197 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7f0674-eab0-4bf3-b1a3-2dd5c77b6b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:36.542895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f0674-eab0-4bf3-b1a3-2dd5c77b6b1b None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:36.543174 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f0674-eab0-4bf3-b1a3-2dd5c77b6b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:36.543460 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f0674-eab0-4bf3-b1a3-2dd5c77b6b1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:36.557344 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7f0674-eab0-4bf3-b1a3-2dd5c77b6b1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:36.559011 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7f0674-eab0-4bf3-b1a3-2dd5c77b6b1b None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:36.576096 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-3e7f0674-eab0-4bf3-b1a3-2dd5c77b6b1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:36.576096 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:36.576096 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:36.576096 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:36.576096 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:36.576096 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:36.576096 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:36.576096 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:36.576096 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:36.576096 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:36.576096 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:36.576096 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:36.576096 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:57:36.576096 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:57:36.576096 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:57:36.577598 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:57:36.577598 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:36.577598 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:36.577598 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:36.577598 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:36.577598 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:36.577598 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:36.577598 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:36.577598 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:36.577598 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:36.577598 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:36.577598 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:36.577598 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:36.577598 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:36.577598 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:36.577598 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:36.577598 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:36.578576 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:57:36.578576 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:57:36.578576 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:36.578576 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:57:36.578576 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 602/1210] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:57:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:36.585451 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7bd6c-0910-4b7f-8153-aa7433deeae4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:36.719838 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:36.723014 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7bd6c-0910-4b7f-8153-aa7433deeae4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:36.770413 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f0e2b0-8e96-4cfb-a06c-1213ac62e86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:36.771099 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0e2b0-8e96-4cfb-a06c-1213ac62e86f None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:36.771339 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0e2b0-8e96-4cfb-a06c-1213ac62e86f None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:36.771604 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0e2b0-8e96-4cfb-a06c-1213ac62e86f None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:36.772359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f0e2b0-8e96-4cfb-a06c-1213ac62e86f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:36.772670 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f0e2b0-8e96-4cfb-a06c-1213ac62e86f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:36.774063 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f0e2b0-8e96-4cfb-a06c-1213ac62e86f None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:36.791423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 609/1211] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:57:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:36.799503 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7f0674-eab0-4bf3-b1a3-2dd5c77b6b1b None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:36.952041 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7f0674-eab0-4bf3-b1a3-2dd5c77b6b1b None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:36.953859 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:37.005692 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e5492-d0e2-426d-86e3-45d9c38a2f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:37.010208 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e5492-d0e2-426d-86e3-45d9c38a2f8b None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:37.010208 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e5492-d0e2-426d-86e3-45d9c38a2f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:37.010208 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e5492-d0e2-426d-86e3-45d9c38a2f8b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:37.020451 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e5492-d0e2-426d-86e3-45d9c38a2f8b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:37.022056 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e5492-d0e2-426d-86e3-45d9c38a2f8b None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:37.037855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-b99e5492-d0e2-426d-86e3-45d9c38a2f8b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 13:57:37.037855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:37.037855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:37.037855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:37.037855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:37.037855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:37.037855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:37.037855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:37.037855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:37.037855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:37.037855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:37.037855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:37.037855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:57:37.037855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:57:37.037855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:57:37.037855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:57:37.039244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:37.039244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:37.039244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:57:37.039244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:57:37.039244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:57:37.039244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:57:37.039244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:37.039244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:37.039244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:37.039244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:37.039244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:37.039244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:37.039244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:37.039244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:37.039244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:37.039244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:37.039244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:37.039244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:37.040336 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:37.040336 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:37.040336 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:57:37.040336 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:57:37.040336 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:57:37.040336 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:57:37.040336 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:57:37.040336 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:57:37.040336 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 13:57:37.040336 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:57:37.040336 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 603/1212] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:57:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:57:37.046694 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f0e2b0-8e96-4cfb-a06c-1213ac62e86f None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:37.181060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f0e2b0-8e96-4cfb-a06c-1213ac62e86f None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:37.219703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d586f95-8402-454d-977d-d39d1b17fcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:37.220565 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d586f95-8402-454d-977d-d39d1b17fcd4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:37.220818 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d586f95-8402-454d-977d-d39d1b17fcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:37.221096 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d586f95-8402-454d-977d-d39d1b17fcd4 None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:37.221968 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d586f95-8402-454d-977d-d39d1b17fcd4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:37.222305 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d586f95-8402-454d-977d-d39d1b17fcd4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:37.223762 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d586f95-8402-454d-977d-d39d1b17fcd4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:37.243830 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 610/1213] 10.222.0.45 () {56 vars in 1143 bytes} [Mon Aug 1 13:57:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:37.249554 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e5492-d0e2-426d-86e3-45d9c38a2f8b None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:37.389004 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e5492-d0e2-426d-86e3-45d9c38a2f8b None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:37.438213 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdac2a2-787b-4a7a-978a-d05bb6d20a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:37.440033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdac2a2-787b-4a7a-978a-d05bb6d20a1b None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:37.440033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdac2a2-787b-4a7a-978a-d05bb6d20a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:37.440558 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdac2a2-787b-4a7a-978a-d05bb6d20a1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:37.492882 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:37.544537 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdac2a2-787b-4a7a-978a-d05bb6d20a1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:37.546141 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdac2a2-787b-4a7a-978a-d05bb6d20a1b None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:37.704436 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 604/1214] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:57:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:57:37.707328 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d586f95-8402-454d-977d-d39d1b17fcd4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:37.739297 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:37.872301 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d586f95-8402-454d-977d-d39d1b17fcd4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:37.927266 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c27f9-d70a-42e2-9541-68688fb3bdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:37.929316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c27f9-d70a-42e2-9541-68688fb3bdf4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:37.929316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c27f9-d70a-42e2-9541-68688fb3bdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:37.929316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c27f9-d70a-42e2-9541-68688fb3bdf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:37.941721 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c27f9-d70a-42e2-9541-68688fb3bdf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:37.943335 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c27f9-d70a-42e2-9541-68688fb3bdf4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:37.966787 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-439c27f9-d70a-42e2-9541-68688fb3bdf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:37.966787 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:37.966787 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:37.966787 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:37.966787 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:37.966787 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:37.966787 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:37.966787 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:37.966787 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:37.966787 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:37.966787 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:37.966787 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:37.966787 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:57:37.966787 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:57:37.966787 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:57:37.967751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:57:37.967751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:37.967751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:37.967751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:37.967751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:37.967751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:37.967751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:37.967751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:37.967751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:37.967751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:37.967751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:37.967751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:37.967751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:37.967751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:37.967751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:37.967751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:37.967751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:37.969359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:57:37.969359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:57:37.969359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:37.969359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:57:37.969359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 611/1215] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:57:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:37.972830 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdac2a2-787b-4a7a-978a-d05bb6d20a1b None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:38.059667 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:38.114653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdac2a2-787b-4a7a-978a-d05bb6d20a1b None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:38.152334 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e4bf7-0d36-4b08-a305-23a71435d2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:38.153034 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e4bf7-0d36-4b08-a305-23a71435d2fd None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:38.153306 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e4bf7-0d36-4b08-a305-23a71435d2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:38.153581 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e4bf7-0d36-4b08-a305-23a71435d2fd None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:38.154394 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e4bf7-0d36-4b08-a305-23a71435d2fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:38.154676 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e4bf7-0d36-4b08-a305-23a71435d2fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:38.156254 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e4bf7-0d36-4b08-a305-23a71435d2fd None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:38.174818 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 605/1216] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:57:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:57:38.180389 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c27f9-d70a-42e2-9541-68688fb3bdf4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:38.282124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:38.329205 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c27f9-d70a-42e2-9541-68688fb3bdf4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:38.375183 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdab1cf1-ca9c-4eed-acee-42507c6c5e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:38.375869 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab1cf1-ca9c-4eed-acee-42507c6c5e34 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:38.376146 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab1cf1-ca9c-4eed-acee-42507c6c5e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:38.376480 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab1cf1-ca9c-4eed-acee-42507c6c5e34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:38.384666 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdab1cf1-ca9c-4eed-acee-42507c6c5e34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:38.386157 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdab1cf1-ca9c-4eed-acee-42507c6c5e34 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:38.399316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-cdab1cf1-ca9c-4eed-acee-42507c6c5e34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:57:38.399316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:38.399316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:38.399316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:38.399316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:38.399316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:38.399316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:38.399316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:38.399316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:38.399316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:38.399316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:38.399316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:38.399316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:57:38.399316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:57:38.399316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:57:38.401252 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:57:38.401252 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:38.401252 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:38.401252 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:38.401252 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:38.401252 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:38.401252 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:38.401252 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:38.401252 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:38.401252 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:38.401252 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:38.401252 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:38.401252 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:38.401252 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:38.401252 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:38.401252 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:38.401252 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:38.403334 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:57:38.403334 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:57:38.403334 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:57:38.403334 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:57:38.403334 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:57:38.403334 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:57:38.403334 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:57:38.403334 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:57:38.403334 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 612/1217] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:57:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:38.407775 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e4bf7-0d36-4b08-a305-23a71435d2fd None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:38.553807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e4bf7-0d36-4b08-a305-23a71435d2fd None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:38.605947 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:38.634018 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b43fd-9463-430e-8a4e-c8494a28b5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:38.635052 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b43fd-9463-430e-8a4e-c8494a28b5ac None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:38.635549 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b43fd-9463-430e-8a4e-c8494a28b5ac None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:38.636177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b43fd-9463-430e-8a4e-c8494a28b5ac None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:38.637979 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b43fd-9463-430e-8a4e-c8494a28b5ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:38.638656 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b43fd-9463-430e-8a4e-c8494a28b5ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:38.641441 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b43fd-9463-430e-8a4e-c8494a28b5ac None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:38.662948 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 606/1218] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:57:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:38.669398 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdab1cf1-ca9c-4eed-acee-42507c6c5e34 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:38.813436 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdab1cf1-ca9c-4eed-acee-42507c6c5e34 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:38.852620 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:38.859683 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ab779-9db2-4740-b749-c3c47aa04a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:38.860784 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ab779-9db2-4740-b749-c3c47aa04a9e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:38.861094 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ab779-9db2-4740-b749-c3c47aa04a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:38.861380 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ab779-9db2-4740-b749-c3c47aa04a9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:38.862267 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ab779-9db2-4740-b749-c3c47aa04a9e None admin] RBAC: Adding query filter params (role.id=d17972f7fe97462b8f9f08aed9bc9cb5, scope.project.id=36f063d8f65d494a89dbbd72c53417c8, user.id=58380e30789041e4ad9e8652bc628658) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:38.862582 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ab779-9db2-4740-b749-c3c47aa04a9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:38.864430 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ab779-9db2-4740-b749-c3c47aa04a9e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:38.872404 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 613/1219] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:57:38 2022] GET /identity/v3/role_assignments?user.id=58380e30789041e4ad9e8652bc628658&scope.project.id=36f063d8f65d494a89dbbd72c53417c8&role.id=d17972f7fe97462b8f9f08aed9bc9cb5 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:57:41.001702 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca54c52-86a3-468b-bf6b-81aa77e0185b None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:41.002354 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca54c52-86a3-468b-bf6b-81aa77e0185b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:41.002354 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca54c52-86a3-468b-bf6b-81aa77e0185b None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:41.004002 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 607/1220] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:57:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:57:41.010451 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3aeaf0-0fda-4ad5-b4bd-97ff9acf9a95 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:41.011417 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3aeaf0-0fda-4ad5-b4bd-97ff9acf9a95 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:41.011536 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3aeaf0-0fda-4ad5-b4bd-97ff9acf9a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:41.036700 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-fa3aeaf0-0fda-4ad5-b4bd-97ff9acf9a95 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:41.038201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fa3aeaf0-0fda-4ad5-b4bd-97ff9acf9a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:41.128482 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 614/1221] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:57:41.152747 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca54c52-86a3-468b-bf6b-81aa77e0185b None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:41.289358 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca54c52-86a3-468b-bf6b-81aa77e0185b None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:41.342922 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9770ba-8c8c-4d6c-b9d4-e4ccb45e611a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:41.343617 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9770ba-8c8c-4d6c-b9d4-e4ccb45e611a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:41.343899 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9770ba-8c8c-4d6c-b9d4-e4ccb45e611a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:41.344185 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9770ba-8c8c-4d6c-b9d4-e4ccb45e611a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:41.357712 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9770ba-8c8c-4d6c-b9d4-e4ccb45e611a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:57:41.363019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-7a9770ba-8c8c-4d6c-b9d4-e4ccb45e611a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:57:41.363019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:41.363019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:41.363019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:41.363019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:41.363019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:41.363019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:41.363019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:41.363019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:41.363019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:41.363019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:41.363019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:41.363019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:57:41.363019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:57:41.363019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:57:41.363019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:57:41.364178 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:57:41.364178 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:57:41.364178 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:57:41.364178 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:57:41.364178 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:41.364178 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:41.364178 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:41.364178 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:41.364178 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:41.364178 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:41.364178 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:41.364178 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:41.364178 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:41.364178 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:41.364178 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:41.364178 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:41.365132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:41.365132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:41.365132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:41.365132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:41.365132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:57:41.365132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:57:41.365132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:57:41.365132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:57:41.365132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:57:41.365132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:57:41.365132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:57:41.365132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:57:41.365132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 608/1222] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 13:57:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:57:41.370551 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3aeaf0-0fda-4ad5-b4bd-97ff9acf9a95 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:41.504746 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3aeaf0-0fda-4ad5-b4bd-97ff9acf9a95 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:41.541754 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:41.559901 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a876c6b-8477-44c4-bf34-66fb801ac83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:41.560905 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a876c6b-8477-44c4-bf34-66fb801ac83d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:41.561394 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a876c6b-8477-44c4-bf34-66fb801ac83d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:41.562025 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a876c6b-8477-44c4-bf34-66fb801ac83d None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:41.562994 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a876c6b-8477-44c4-bf34-66fb801ac83d None admin] RBAC: Adding query filter params (name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:41.563463 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a876c6b-8477-44c4-bf34-66fb801ac83d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:41.565548 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a876c6b-8477-44c4-bf34-66fb801ac83d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:41.577786 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 615/1223] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 13:57:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:41.585697 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9770ba-8c8c-4d6c-b9d4-e4ccb45e611a None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:41.657827 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:41.744787 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9770ba-8c8c-4d6c-b9d4-e4ccb45e611a None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:41.816186 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a0690-7535-4f2e-a88e-575167d9d543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:41.817332 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a0690-7535-4f2e-a88e-575167d9d543 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:41.817763 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a0690-7535-4f2e-a88e-575167d9d543 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:41.818169 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a0690-7535-4f2e-a88e-575167d9d543 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:41.841065 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a0690-7535-4f2e-a88e-575167d9d543 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:41.842642 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a0690-7535-4f2e-a88e-575167d9d543 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:41.862031 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-a69a0690-7535-4f2e-a88e-575167d9d543 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:41.862031 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:41.862031 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:41.862031 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:41.862031 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:41.862031 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:41.862031 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:41.862031 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:41.862031 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:41.862031 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:41.862031 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:41.862031 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:41.862031 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:57:41.862031 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:57:41.862031 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:57:41.864142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:57:41.864142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:41.864142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:41.864142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:41.864142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:41.864142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:41.864142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:41.864142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:41.864142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:41.864142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:41.864142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:41.864142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:41.864142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:41.864142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:41.864142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:41.864142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:41.864142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:41.865389 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:57:41.865389 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:57:41.865389 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:41.865389 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:57:41.865389 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 609/1224] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:57:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:41.869768 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a876c6b-8477-44c4-bf34-66fb801ac83d None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:41.996710 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a876c6b-8477-44c4-bf34-66fb801ac83d None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:42.048796 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829544a8-af4a-4576-af1c-4b89530866b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:42.049942 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829544a8-af4a-4576-af1c-4b89530866b8 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:42.049942 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829544a8-af4a-4576-af1c-4b89530866b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:42.050210 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829544a8-af4a-4576-af1c-4b89530866b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:42.050837 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829544a8-af4a-4576-af1c-4b89530866b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:42.051143 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829544a8-af4a-4576-af1c-4b89530866b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:42.052591 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829544a8-af4a-4576-af1c-4b89530866b8 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:42.068418 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 616/1225] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:57:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:42.074955 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a0690-7535-4f2e-a88e-575167d9d543 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:42.210550 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:42.219426 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a0690-7535-4f2e-a88e-575167d9d543 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:42.261627 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a75069e-0f68-4167-a457-6f0bcf171281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:42.262288 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75069e-0f68-4167-a457-6f0bcf171281 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:42.262565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75069e-0f68-4167-a457-6f0bcf171281 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:42.262836 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75069e-0f68-4167-a457-6f0bcf171281 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:42.271525 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a75069e-0f68-4167-a457-6f0bcf171281 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:42.273365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a75069e-0f68-4167-a457-6f0bcf171281 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:42.288499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-9a75069e-0f68-4167-a457-6f0bcf171281 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 13:57:42.288499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:42.288499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:42.288499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:42.288499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:42.288499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:42.288499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:42.288499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:42.288499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:42.288499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:42.288499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:42.288499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:42.288499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:57:42.288499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:57:42.288499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:57:42.288499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:57:42.289698 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:42.289698 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:42.289698 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:57:42.289698 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:57:42.289698 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:57:42.289698 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:57:42.289698 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:42.289698 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:42.289698 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:42.289698 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:42.289698 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:42.289698 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:42.289698 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:42.289698 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:42.289698 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:42.289698 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:42.289698 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:42.289698 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:42.290880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:42.290880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:42.290880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:57:42.290880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:57:42.290880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:57:42.290880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:57:42.290880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:57:42.290880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:57:42.290880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 13:57:42.290880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:57:42.290880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 610/1226] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:57:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:42.296195 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829544a8-af4a-4576-af1c-4b89530866b8 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:42.351100 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:42.444253 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829544a8-af4a-4576-af1c-4b89530866b8 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:42.494848 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef28316-e144-40c1-8d0f-65a6d166d1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:42.495531 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef28316-e144-40c1-8d0f-65a6d166d1fb None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:42.495809 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef28316-e144-40c1-8d0f-65a6d166d1fb None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:42.496302 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef28316-e144-40c1-8d0f-65a6d166d1fb None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:42.497147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef28316-e144-40c1-8d0f-65a6d166d1fb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:42.498415 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef28316-e144-40c1-8d0f-65a6d166d1fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:42.499066 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef28316-e144-40c1-8d0f-65a6d166d1fb None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:42.531108 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 617/1227] 10.222.0.45 () {56 vars in 1143 bytes} [Mon Aug 1 13:57:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:42.533305 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a75069e-0f68-4167-a457-6f0bcf171281 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:42.662019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a75069e-0f68-4167-a457-6f0bcf171281 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:42.713057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3826d8d3-ef5f-4d7b-8e7a-66311265fbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:42.713830 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826d8d3-ef5f-4d7b-8e7a-66311265fbc2 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:42.714117 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826d8d3-ef5f-4d7b-8e7a-66311265fbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:42.714377 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826d8d3-ef5f-4d7b-8e7a-66311265fbc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:42.727029 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3826d8d3-ef5f-4d7b-8e7a-66311265fbc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:42.728584 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3826d8d3-ef5f-4d7b-8e7a-66311265fbc2 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:42.744453 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-3826d8d3-ef5f-4d7b-8e7a-66311265fbc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:42.744453 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:42.744453 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:42.744453 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:42.744453 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:42.744453 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:42.744453 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:42.744453 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:42.744453 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:42.744453 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:42.744453 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:42.744453 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:42.744453 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:57:42.744453 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:57:42.744453 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:57:42.745626 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:57:42.745626 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:42.745626 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:42.745626 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:42.745626 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:42.745626 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:42.745626 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:42.745626 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:42.745626 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:42.745626 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:42.745626 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:42.745626 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:42.745626 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:42.745626 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:42.745626 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:42.745626 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:42.745626 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:42.746798 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:57:42.746798 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:57:42.746798 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:42.746798 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:57:42.746798 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 611/1228] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:57:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:42.756444 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef28316-e144-40c1-8d0f-65a6d166d1fb None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:42.909044 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef28316-e144-40c1-8d0f-65a6d166d1fb None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:42.911716 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:42.949682 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a83aa-2b89-40bf-ace9-c9c4a30433f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:42.950234 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a83aa-2b89-40bf-ace9-c9c4a30433f9 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:42.950497 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a83aa-2b89-40bf-ace9-c9c4a30433f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:42.950776 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a83aa-2b89-40bf-ace9-c9c4a30433f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:42.951478 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a83aa-2b89-40bf-ace9-c9c4a30433f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:42.951850 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a83aa-2b89-40bf-ace9-c9c4a30433f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:42.953404 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a83aa-2b89-40bf-ace9-c9c4a30433f9 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:42.966223 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 618/1229] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:57:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:42.972557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3826d8d3-ef5f-4d7b-8e7a-66311265fbc2 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:42.984251 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:43.095111 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3826d8d3-ef5f-4d7b-8e7a-66311265fbc2 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:43.149879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c31d5-6a59-4616-ba97-6b0381e75103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:43.150910 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c31d5-6a59-4616-ba97-6b0381e75103 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:43.151336 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c31d5-6a59-4616-ba97-6b0381e75103 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:43.151618 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c31d5-6a59-4616-ba97-6b0381e75103 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:43.164413 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c31d5-6a59-4616-ba97-6b0381e75103 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:43.165938 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c31d5-6a59-4616-ba97-6b0381e75103 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:43.180585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-410c31d5-6a59-4616-ba97-6b0381e75103 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:57:43.180585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:43.180585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:43.180585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:43.180585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:43.180585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:43.180585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:43.180585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:43.180585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:43.180585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:43.180585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:43.180585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:43.180585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:57:43.180585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:57:43.180585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:57:43.182406 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:57:43.182406 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:43.182406 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:43.182406 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:43.182406 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:43.182406 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:43.182406 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:43.182406 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:43.182406 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:43.182406 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:43.182406 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:43.182406 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:43.182406 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:43.182406 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:43.182406 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:43.182406 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:43.182406 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:43.184206 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:57:43.184206 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:57:43.184206 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:57:43.184206 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:57:43.184206 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:57:43.184206 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:57:43.184206 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:57:43.184206 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:57:43.184206 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 612/1230] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:57:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:43.188153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a83aa-2b89-40bf-ace9-c9c4a30433f9 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:43.342575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a83aa-2b89-40bf-ace9-c9c4a30433f9 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:43.383461 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103ca15-a4a8-49e5-b6d0-bdfaecd80e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:43.384091 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103ca15-a4a8-49e5-b6d0-bdfaecd80e58 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:43.384444 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103ca15-a4a8-49e5-b6d0-bdfaecd80e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:43.384768 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103ca15-a4a8-49e5-b6d0-bdfaecd80e58 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:43.385526 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3103ca15-a4a8-49e5-b6d0-bdfaecd80e58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:43.385872 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3103ca15-a4a8-49e5-b6d0-bdfaecd80e58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:43.387297 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3103ca15-a4a8-49e5-b6d0-bdfaecd80e58 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:43.400499 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 619/1231] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:57:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:43.408046 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c31d5-6a59-4616-ba97-6b0381e75103 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:43.498632 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:43.546218 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c31d5-6a59-4616-ba97-6b0381e75103 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:43.587276 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6029f6e5-e2b9-4b7e-a0ac-9bce3c312e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:43.588766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6029f6e5-e2b9-4b7e-a0ac-9bce3c312e61 None admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:43.589055 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6029f6e5-e2b9-4b7e-a0ac-9bce3c312e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:43.589392 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6029f6e5-e2b9-4b7e-a0ac-9bce3c312e61 None admin] PATH_INFO: `/v3/projects/36f063d8f65d494a89dbbd72c53417c8/users/58380e30789041e4ad9e8652bc628658/roles/d17972f7fe97462b8f9f08aed9bc9cb5` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:43.620054 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6029f6e5-e2b9-4b7e-a0ac-9bce3c312e61 None admin] RBAC: Authorizing `identity:create_grant(project_id=36f063d8f65d494a89dbbd72c53417c8, user_id=58380e30789041e4ad9e8652bc628658, role_id=d17972f7fe97462b8f9f08aed9bc9cb5)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:43.623011 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6029f6e5-e2b9-4b7e-a0ac-9bce3c312e61 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:43.638921 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 613/1232] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 13:57:43 2022] PUT /identity/v3/projects/36f063d8f65d494a89dbbd72c53417c8/users/58380e30789041e4ad9e8652bc628658/roles/d17972f7fe97462b8f9f08aed9bc9cb5 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:57:45.893397 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a9415-1542-4a4c-bfe6-42b8bda1f387 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:45.894055 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a9415-1542-4a4c-bfe6-42b8bda1f387 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:45.894055 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a9415-1542-4a4c-bfe6-42b8bda1f387 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:45.895219 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 620/1233] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:57:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:57:45.902742 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01483745-30c7-4326-a046-bf310e0a4f41 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:45.903441 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01483745-30c7-4326-a046-bf310e0a4f41 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:45.903863 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01483745-30c7-4326-a046-bf310e0a4f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:45.930227 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-01483745-30c7-4326-a046-bf310e0a4f41 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:45.932084 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-01483745-30c7-4326-a046-bf310e0a4f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:46.023959 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 614/1234] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:57:46.048968 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a9415-1542-4a4c-bfe6-42b8bda1f387 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:46.078192 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:46.191531 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a9415-1542-4a4c-bfe6-42b8bda1f387 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:46.236126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa34020-1592-4092-8502-b0755fa3fab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:46.237098 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa34020-1592-4092-8502-b0755fa3fab7 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:46.237630 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa34020-1592-4092-8502-b0755fa3fab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:46.238118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa34020-1592-4092-8502-b0755fa3fab7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:46.255081 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa34020-1592-4092-8502-b0755fa3fab7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:57:46.260990 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-aaa34020-1592-4092-8502-b0755fa3fab7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:57:46.260990 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:46.260990 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:46.260990 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:46.260990 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:46.260990 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:46.260990 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:46.260990 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:46.260990 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:46.260990 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:46.260990 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:46.260990 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:46.260990 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:57:46.260990 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:57:46.260990 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:57:46.260990 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:57:46.263201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:57:46.263201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:57:46.263201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:57:46.263201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:57:46.263201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:46.263201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:46.263201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:46.263201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:46.263201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:46.263201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:46.263201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:46.263201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:46.263201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:46.263201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:46.263201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:46.263201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:46.270771 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:46.270771 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:46.270771 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:46.270771 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:46.270771 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:57:46.270771 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:57:46.270771 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:57:46.270771 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:57:46.270771 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:57:46.270771 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:57:46.270771 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:57:46.270771 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:57:46.270771 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 621/1235] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 13:57:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:57:46.271570 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01483745-30c7-4326-a046-bf310e0a4f41 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:46.334723 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:46.413031 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01483745-30c7-4326-a046-bf310e0a4f41 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:46.452951 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ffc61b-042c-4f9e-85ba-165c7a3bc95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:46.454174 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffc61b-042c-4f9e-85ba-165c7a3bc95a None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:46.454174 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffc61b-042c-4f9e-85ba-165c7a3bc95a None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:46.454174 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffc61b-042c-4f9e-85ba-165c7a3bc95a None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:46.454980 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ffc61b-042c-4f9e-85ba-165c7a3bc95a None admin] RBAC: Adding query filter params (name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:46.455214 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ffc61b-042c-4f9e-85ba-165c7a3bc95a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:46.456618 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ffc61b-042c-4f9e-85ba-165c7a3bc95a None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:46.465424 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 615/1236] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 13:57:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:46.471373 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa34020-1592-4092-8502-b0755fa3fab7 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:46.591917 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:46.616855 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa34020-1592-4092-8502-b0755fa3fab7 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:46.656495 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045eafc-7ac8-4938-b816-5c51b1e1bf08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:46.657151 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045eafc-7ac8-4938-b816-5c51b1e1bf08 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:46.657423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045eafc-7ac8-4938-b816-5c51b1e1bf08 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:46.657694 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045eafc-7ac8-4938-b816-5c51b1e1bf08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:46.669062 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045eafc-7ac8-4938-b816-5c51b1e1bf08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:46.670547 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045eafc-7ac8-4938-b816-5c51b1e1bf08 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:46.687310 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-7045eafc-7ac8-4938-b816-5c51b1e1bf08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:46.687310 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:46.687310 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:46.687310 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:46.687310 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:46.687310 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:46.687310 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:46.687310 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:46.687310 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:46.687310 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:46.687310 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:46.687310 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:46.687310 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:57:46.687310 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:57:46.687310 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:57:46.696245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:57:46.696245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:46.696245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:46.696245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:46.696245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:46.696245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:46.696245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:46.696245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:46.696245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:46.696245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:46.696245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:46.696245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:46.696245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:46.696245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:46.696245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:46.696245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:46.696245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:46.698451 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:57:46.698451 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:57:46.698451 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:46.698451 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:57:46.698451 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 622/1237] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:57:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:46.699073 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ffc61b-042c-4f9e-85ba-165c7a3bc95a None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:46.845968 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ffc61b-042c-4f9e-85ba-165c7a3bc95a None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:46.861888 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:46.897846 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7405b80-fc14-4ffb-b12d-91f149a3e35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:46.898417 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7405b80-fc14-4ffb-b12d-91f149a3e35d None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:46.898696 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7405b80-fc14-4ffb-b12d-91f149a3e35d None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:46.898968 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7405b80-fc14-4ffb-b12d-91f149a3e35d None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:46.899671 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7405b80-fc14-4ffb-b12d-91f149a3e35d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:46.899973 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7405b80-fc14-4ffb-b12d-91f149a3e35d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:46.901407 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7405b80-fc14-4ffb-b12d-91f149a3e35d None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:46.917719 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 616/1238] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:57:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:46.924868 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045eafc-7ac8-4938-b816-5c51b1e1bf08 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:47.054828 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045eafc-7ac8-4938-b816-5c51b1e1bf08 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:47.096642 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9d23d-ad39-4aaa-abd1-15c1ac1dffaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:47.097222 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9d23d-ad39-4aaa-abd1-15c1ac1dffaf None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:47.097500 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9d23d-ad39-4aaa-abd1-15c1ac1dffaf None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:47.097784 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9d23d-ad39-4aaa-abd1-15c1ac1dffaf None admin] PATH_INFO: `/v3/users/placement` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:47.101537 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:47.108320 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c9d23d-ad39-4aaa-abd1-15c1ac1dffaf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:47.110241 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c9d23d-ad39-4aaa-abd1-15c1ac1dffaf None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:47.131037 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-08c9d23d-ad39-4aaa-abd1-15c1ac1dffaf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 13:57:47.131037 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:47.131037 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:47.131037 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:47.131037 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:47.131037 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:47.131037 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:47.131037 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:47.131037 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:47.131037 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:47.131037 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:47.131037 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:47.131037 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:57:47.131037 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:57:47.131037 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:57:47.131037 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:57:47.133549 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:47.133549 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:47.133549 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:57:47.133549 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:57:47.133549 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:57:47.133549 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:57:47.133549 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:47.133549 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:47.133549 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:47.133549 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:47.133549 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:47.133549 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:47.133549 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:47.133549 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:47.133549 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:47.133549 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:47.133549 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:47.133549 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:47.134915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:47.134915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:47.134915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:57:47.134915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:57:47.134915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:57:47.134915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:57:47.134915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:57:47.134915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:57:47.134915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 13:57:47.134915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:57:47.134915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 623/1239] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:57:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:47.139284 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7405b80-fc14-4ffb-b12d-91f149a3e35d None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:47.283143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7405b80-fc14-4ffb-b12d-91f149a3e35d None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:47.334574 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b147b-f51c-41f1-b16b-8a2352cb2438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:47.335218 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b147b-f51c-41f1-b16b-8a2352cb2438 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:47.335489 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b147b-f51c-41f1-b16b-8a2352cb2438 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:47.335756 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b147b-f51c-41f1-b16b-8a2352cb2438 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:47.336604 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674b147b-f51c-41f1-b16b-8a2352cb2438 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:47.336907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674b147b-f51c-41f1-b16b-8a2352cb2438 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:47.338369 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674b147b-f51c-41f1-b16b-8a2352cb2438 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:47.364112 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 617/1240] 10.222.0.45 () {56 vars in 1143 bytes} [Mon Aug 1 13:57:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:57:47.370283 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9d23d-ad39-4aaa-abd1-15c1ac1dffaf None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:47.533327 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9d23d-ad39-4aaa-abd1-15c1ac1dffaf None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:47.584020 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1698c0f-4376-4eda-90ce-82918c90e6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:47.584698 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1698c0f-4376-4eda-90ce-82918c90e6d3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:47.584979 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1698c0f-4376-4eda-90ce-82918c90e6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:47.585454 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1698c0f-4376-4eda-90ce-82918c90e6d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:47.678000 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:47.690877 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1698c0f-4376-4eda-90ce-82918c90e6d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:47.692624 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1698c0f-4376-4eda-90ce-82918c90e6d3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:47.832870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 624/1241] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:57:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:57:47.839758 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b147b-f51c-41f1-b16b-8a2352cb2438 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:47.885637 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:47.997739 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b147b-f51c-41f1-b16b-8a2352cb2438 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:48.045770 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81f49c-41a8-41f6-878f-9701ee145eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:48.046450 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81f49c-41a8-41f6-878f-9701ee145eed None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:48.046710 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81f49c-41a8-41f6-878f-9701ee145eed None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:48.047019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81f49c-41a8-41f6-878f-9701ee145eed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:48.064018 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b81f49c-41a8-41f6-878f-9701ee145eed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:48.065802 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b81f49c-41a8-41f6-878f-9701ee145eed None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:48.084227 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-1b81f49c-41a8-41f6-878f-9701ee145eed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:48.084227 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:48.084227 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:48.084227 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:48.084227 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:48.084227 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:48.084227 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:48.084227 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:48.084227 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:48.084227 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:48.084227 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:48.084227 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:48.084227 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:57:48.084227 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:57:48.084227 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:57:48.085565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:57:48.085565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:48.085565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:48.085565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:48.085565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:48.085565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:48.085565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:48.085565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:48.085565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:48.085565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:48.085565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:48.085565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:48.085565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:48.085565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:48.085565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:48.085565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:48.085565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:48.088006 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:57:48.088006 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:57:48.088006 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:48.088006 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:57:48.088006 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 618/1242] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:57:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:48.094371 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1698c0f-4376-4eda-90ce-82918c90e6d3 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:48.238864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:48.245945 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1698c0f-4376-4eda-90ce-82918c90e6d3 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:48.293888 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b6bd81-3ccb-43ea-8e6a-da5e3ab7262e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:48.294432 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6bd81-3ccb-43ea-8e6a-da5e3ab7262e None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:48.294712 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6bd81-3ccb-43ea-8e6a-da5e3ab7262e None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:48.294976 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6bd81-3ccb-43ea-8e6a-da5e3ab7262e None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:48.295673 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b6bd81-3ccb-43ea-8e6a-da5e3ab7262e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:48.295969 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b6bd81-3ccb-43ea-8e6a-da5e3ab7262e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:48.297391 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b6bd81-3ccb-43ea-8e6a-da5e3ab7262e None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:48.312034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 625/1243] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:57:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:57:48.318223 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81f49c-41a8-41f6-878f-9701ee145eed None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:48.416038 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:48.445217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81f49c-41a8-41f6-878f-9701ee145eed None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:48.492794 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02a457-95bf-4dac-a042-e894cdaa9eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:48.493972 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02a457-95bf-4dac-a042-e894cdaa9eff None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:48.494572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02a457-95bf-4dac-a042-e894cdaa9eff None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:48.494886 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02a457-95bf-4dac-a042-e894cdaa9eff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:48.505218 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b02a457-95bf-4dac-a042-e894cdaa9eff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:48.506795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b02a457-95bf-4dac-a042-e894cdaa9eff None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:48.522015 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-0b02a457-95bf-4dac-a042-e894cdaa9eff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:57:48.522015 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:48.522015 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:48.522015 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:48.522015 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:48.522015 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:48.522015 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:48.522015 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:48.522015 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:48.522015 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:48.522015 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:48.522015 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:48.522015 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:57:48.522015 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:57:48.522015 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:57:48.523259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:57:48.523259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:48.523259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:48.523259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:48.523259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:48.523259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:48.523259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:48.523259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:48.523259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:48.523259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:48.523259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:48.523259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:48.523259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:48.523259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:48.523259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:48.523259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:48.523259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:48.524352 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:57:48.524352 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:57:48.524352 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:57:48.524352 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:57:48.524352 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:57:48.524352 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:57:48.524352 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:57:48.524352 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:57:48.524352 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 619/1244] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:57:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:48.529969 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b6bd81-3ccb-43ea-8e6a-da5e3ab7262e None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:48.672216 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b6bd81-3ccb-43ea-8e6a-da5e3ab7262e None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:48.726782 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8b9ee-57d5-4d66-977f-bafd5d8a9603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:48.727661 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8b9ee-57d5-4d66-977f-bafd5d8a9603 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:48.727894 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8b9ee-57d5-4d66-977f-bafd5d8a9603 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:48.728208 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8b9ee-57d5-4d66-977f-bafd5d8a9603 None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:48.729036 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8b9ee-57d5-4d66-977f-bafd5d8a9603 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:48.729349 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8b9ee-57d5-4d66-977f-bafd5d8a9603 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:48.731106 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8b9ee-57d5-4d66-977f-bafd5d8a9603 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:48.747626 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 626/1245] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:57:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:48.754922 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02a457-95bf-4dac-a042-e894cdaa9eff None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:48.901893 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02a457-95bf-4dac-a042-e894cdaa9eff None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:48.947066 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deab190-4ce2-448e-a811-8196a1343ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:48.947916 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deab190-4ce2-448e-a811-8196a1343ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:48.948387 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deab190-4ce2-448e-a811-8196a1343ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:48.948721 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deab190-4ce2-448e-a811-8196a1343ad8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:48.949726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deab190-4ce2-448e-a811-8196a1343ad8 None admin] RBAC: Adding query filter params (role.id=d17972f7fe97462b8f9f08aed9bc9cb5, scope.project.id=36f063d8f65d494a89dbbd72c53417c8, user.id=58380e30789041e4ad9e8652bc628658) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:48.950083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deab190-4ce2-448e-a811-8196a1343ad8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:48.951651 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deab190-4ce2-448e-a811-8196a1343ad8 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:48.962828 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 620/1246] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:57:48 2022] GET /identity/v3/role_assignments?user.id=58380e30789041e4ad9e8652bc628658&scope.project.id=36f063d8f65d494a89dbbd72c53417c8&role.id=d17972f7fe97462b8f9f08aed9bc9cb5 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:57:51.274762 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fa661-085f-42c2-b300-067761521b45 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:51.275515 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fa661-085f-42c2-b300-067761521b45 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:51.275515 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fa661-085f-42c2-b300-067761521b45 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:51.276795 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 627/1247] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:57:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:57:51.283588 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34586e4-41d0-403b-a99e-850e089e4eeb None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:51.284332 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34586e4-41d0-403b-a99e-850e089e4eeb None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:51.284543 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34586e4-41d0-403b-a99e-850e089e4eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:51.292314 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:51.310421 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-a34586e4-41d0-403b-a99e-850e089e4eeb None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:51.311766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a34586e4-41d0-403b-a99e-850e089e4eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:51.400207 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 621/1248] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:57:51.428042 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25fa661-085f-42c2-b300-067761521b45 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:51.451524 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:51.573472 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25fa661-085f-42c2-b300-067761521b45 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:51.612462 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea1b87-8606-4c54-aafd-38281c59adff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:51.613307 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea1b87-8606-4c54-aafd-38281c59adff None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:51.613614 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea1b87-8606-4c54-aafd-38281c59adff None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:51.613934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea1b87-8606-4c54-aafd-38281c59adff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:51.629251 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ea1b87-8606-4c54-aafd-38281c59adff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:57:51.635186 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-b1ea1b87-8606-4c54-aafd-38281c59adff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:57:51.635186 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:51.635186 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:51.635186 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:51.635186 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:51.635186 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:51.635186 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:51.635186 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:51.635186 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:51.635186 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:51.635186 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:51.635186 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:51.635186 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:57:51.635186 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:57:51.635186 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:57:51.635186 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:57:51.636366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:57:51.636366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:57:51.636366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:57:51.636366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:57:51.636366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:51.636366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:51.636366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:51.636366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:51.636366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:51.636366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:51.636366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:51.636366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:51.636366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:51.636366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:51.636366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:51.636366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:51.637454 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:51.637454 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:51.637454 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:51.637454 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:51.637454 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:57:51.637454 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:57:51.637454 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:57:51.637454 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:57:51.637454 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:57:51.637454 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:57:51.637454 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:57:51.637454 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:57:51.644314 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 628/1249] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:57:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:57:51.644472 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34586e4-41d0-403b-a99e-850e089e4eeb None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:51.779954 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34586e4-41d0-403b-a99e-850e089e4eeb None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:51.823089 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:51.830027 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b042f7-1f27-4e67-8547-3baa3ea8b175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:51.830707 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b042f7-1f27-4e67-8547-3baa3ea8b175 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:51.830988 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b042f7-1f27-4e67-8547-3baa3ea8b175 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:51.831256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b042f7-1f27-4e67-8547-3baa3ea8b175 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:51.832008 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b042f7-1f27-4e67-8547-3baa3ea8b175 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:51.832340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b042f7-1f27-4e67-8547-3baa3ea8b175 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:51.833947 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b042f7-1f27-4e67-8547-3baa3ea8b175 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:51.844362 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 622/1250] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:57:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:51.850701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea1b87-8606-4c54-aafd-38281c59adff None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:51.960483 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:51.990906 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea1b87-8606-4c54-aafd-38281c59adff None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:52.034332 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f933d688-2fd3-45d4-8f59-092dbffb2b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:52.035106 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933d688-2fd3-45d4-8f59-092dbffb2b43 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:52.035384 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933d688-2fd3-45d4-8f59-092dbffb2b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:52.035727 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933d688-2fd3-45d4-8f59-092dbffb2b43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:52.048951 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f933d688-2fd3-45d4-8f59-092dbffb2b43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:52.051056 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f933d688-2fd3-45d4-8f59-092dbffb2b43 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:52.075088 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-f933d688-2fd3-45d4-8f59-092dbffb2b43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:52.075088 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:52.075088 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:52.075088 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:52.075088 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:52.075088 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:52.075088 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:52.075088 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:52.075088 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:52.075088 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:52.075088 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:52.075088 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:52.075088 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:57:52.075088 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:57:52.075088 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:57:52.077282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:57:52.077282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:52.077282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:52.077282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:52.077282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:52.077282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:52.077282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:52.077282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:52.077282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:52.077282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:52.077282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:52.077282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:52.077282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:52.077282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:52.077282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:52.077282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:52.077282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:52.079519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:57:52.079519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:57:52.079519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:52.079519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:57:52.082383 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 629/1251] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:57:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:52.090823 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b042f7-1f27-4e67-8547-3baa3ea8b175 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:52.246381 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b042f7-1f27-4e67-8547-3baa3ea8b175 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:52.286610 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf98b1-f1ce-4c67-91e5-c02dd0a1b180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:52.287658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf98b1-f1ce-4c67-91e5-c02dd0a1b180 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:52.287658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf98b1-f1ce-4c67-91e5-c02dd0a1b180 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:52.287971 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf98b1-f1ce-4c67-91e5-c02dd0a1b180 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:52.288677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbf98b1-f1ce-4c67-91e5-c02dd0a1b180 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:52.288986 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbf98b1-f1ce-4c67-91e5-c02dd0a1b180 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:52.290441 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbf98b1-f1ce-4c67-91e5-c02dd0a1b180 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:52.304524 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 623/1252] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:57:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:57:52.310801 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f933d688-2fd3-45d4-8f59-092dbffb2b43 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:52.452454 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f933d688-2fd3-45d4-8f59-092dbffb2b43 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:52.502989 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:52.517921 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f453aaa-4969-4512-bbe6-9974a133c133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:52.518962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f453aaa-4969-4512-bbe6-9974a133c133 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:52.519952 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f453aaa-4969-4512-bbe6-9974a133c133 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:52.519952 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f453aaa-4969-4512-bbe6-9974a133c133 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:52.534446 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f453aaa-4969-4512-bbe6-9974a133c133 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:52.537233 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f453aaa-4969-4512-bbe6-9974a133c133 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:52.554938 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-2f453aaa-4969-4512-bbe6-9974a133c133 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 13:57:52.554938 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:52.554938 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:52.554938 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:52.554938 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:52.554938 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:52.554938 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:52.554938 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:52.554938 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:52.554938 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:52.554938 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:52.554938 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:52.554938 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:57:52.554938 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:57:52.554938 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:57:52.554938 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:57:52.557483 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:52.557483 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:52.557483 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:57:52.557483 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:57:52.557483 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:57:52.557483 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:57:52.557483 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:52.557483 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:52.557483 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:52.557483 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:52.557483 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:52.557483 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:52.557483 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:52.557483 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:52.557483 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:52.557483 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:52.557483 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:52.557483 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:52.559534 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:52.559534 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:52.559534 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:57:52.559534 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:57:52.559534 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:57:52.559534 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:57:52.559534 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:57:52.559534 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:57:52.559534 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 13:57:52.559534 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:57:52.564041 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 630/1253] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:57:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:57:52.565912 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf98b1-f1ce-4c67-91e5-c02dd0a1b180 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:52.644174 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:52.711240 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf98b1-f1ce-4c67-91e5-c02dd0a1b180 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:52.776405 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c2ad9c-fad2-4322-bdd3-dcbdfd924f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:52.777789 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2ad9c-fad2-4322-bdd3-dcbdfd924f87 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:52.778262 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2ad9c-fad2-4322-bdd3-dcbdfd924f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:52.778721 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2ad9c-fad2-4322-bdd3-dcbdfd924f87 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:52.780175 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c2ad9c-fad2-4322-bdd3-dcbdfd924f87 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:52.780734 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c2ad9c-fad2-4322-bdd3-dcbdfd924f87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:52.783468 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c2ad9c-fad2-4322-bdd3-dcbdfd924f87 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:52.812911 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 624/1254] 10.222.0.45 () {56 vars in 1143 bytes} [Mon Aug 1 13:57:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:52.820932 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f453aaa-4969-4512-bbe6-9974a133c133 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:52.962093 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f453aaa-4969-4512-bbe6-9974a133c133 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:53.012215 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e848aea-ea42-471d-8195-27e5f5401620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:53.012986 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e848aea-ea42-471d-8195-27e5f5401620 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:53.013299 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e848aea-ea42-471d-8195-27e5f5401620 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:53.013650 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e848aea-ea42-471d-8195-27e5f5401620 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:53.086299 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:53.106634 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e848aea-ea42-471d-8195-27e5f5401620 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:53.108109 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e848aea-ea42-471d-8195-27e5f5401620 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:53.237070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 631/1255] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:57:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:57:53.244942 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c2ad9c-fad2-4322-bdd3-dcbdfd924f87 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:53.383275 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c2ad9c-fad2-4322-bdd3-dcbdfd924f87 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:53.400146 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:53.431895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e106b389-1450-4bbb-897c-0a3b3d4a1831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:53.432844 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106b389-1450-4bbb-897c-0a3b3d4a1831 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:53.433275 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106b389-1450-4bbb-897c-0a3b3d4a1831 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:53.433706 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106b389-1450-4bbb-897c-0a3b3d4a1831 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:53.449509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e106b389-1450-4bbb-897c-0a3b3d4a1831 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:53.451589 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e106b389-1450-4bbb-897c-0a3b3d4a1831 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:53.472256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-e106b389-1450-4bbb-897c-0a3b3d4a1831 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:53.472256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:53.472256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:53.472256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:53.472256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:53.472256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:53.472256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:53.472256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:53.472256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:53.472256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:53.472256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:53.472256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:53.472256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:57:53.472256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:57:53.472256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:57:53.473714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:57:53.473714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:53.473714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:53.473714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:53.473714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:53.473714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:53.473714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:53.473714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:53.473714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:53.473714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:53.473714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:53.473714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:53.473714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:53.473714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:53.473714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:53.473714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:53.473714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:53.475201 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:57:53.475201 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:57:53.475201 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:53.475201 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:57:53.475201 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 625/1256] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:57:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:57:53.481128 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e848aea-ea42-471d-8195-27e5f5401620 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:53.623138 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:53.644034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e848aea-ea42-471d-8195-27e5f5401620 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:53.705222 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c8fc6-f3bb-46c6-b6d3-36d3ab8c33dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:53.708800 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c8fc6-f3bb-46c6-b6d3-36d3ab8c33dd None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:53.709073 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c8fc6-f3bb-46c6-b6d3-36d3ab8c33dd None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:53.709389 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c8fc6-f3bb-46c6-b6d3-36d3ab8c33dd None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:53.710637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6c8fc6-f3bb-46c6-b6d3-36d3ab8c33dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:53.710637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6c8fc6-f3bb-46c6-b6d3-36d3ab8c33dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:53.712212 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6c8fc6-f3bb-46c6-b6d3-36d3ab8c33dd None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:53.736585 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 632/1257] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:57:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:53.742938 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e106b389-1450-4bbb-897c-0a3b3d4a1831 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:53.890867 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e106b389-1450-4bbb-897c-0a3b3d4a1831 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:53.941258 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f287fc-9c8a-4cf5-838a-ce60b8ff6cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:53.941936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f287fc-9c8a-4cf5-838a-ce60b8ff6cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:53.942294 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f287fc-9c8a-4cf5-838a-ce60b8ff6cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:53.942788 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f287fc-9c8a-4cf5-838a-ce60b8ff6cb9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:53.953960 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f287fc-9c8a-4cf5-838a-ce60b8ff6cb9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:53.956380 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f287fc-9c8a-4cf5-838a-ce60b8ff6cb9 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:53.970694 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-d7f287fc-9c8a-4cf5-838a-ce60b8ff6cb9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:57:53.970694 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:53.970694 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:53.970694 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:53.970694 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:53.970694 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:53.970694 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:53.970694 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:53.970694 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:53.970694 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:53.970694 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:53.970694 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:53.970694 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:57:53.970694 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:57:53.970694 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:57:53.972934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:57:53.972934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:53.972934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:53.972934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:53.972934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:53.972934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:53.972934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:53.972934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:53.972934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:53.972934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:53.972934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:53.972934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:53.972934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:53.972934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:53.972934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:53.972934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:53.972934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:53.975033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:57:53.975033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:57:53.975033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:57:53.975033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:57:53.975033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:57:53.975033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:57:53.975033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:57:53.975033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:57:53.975033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 626/1258] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:57:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:53.978518 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c8fc6-f3bb-46c6-b6d3-36d3ab8c33dd None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:54.148439 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c8fc6-f3bb-46c6-b6d3-36d3ab8c33dd None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:54.191823 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c946f-d4b2-420a-ae82-4b5cd77c3c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:54.192484 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c946f-d4b2-420a-ae82-4b5cd77c3c8d None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:54.192754 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c946f-d4b2-420a-ae82-4b5cd77c3c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:54.193021 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c946f-d4b2-420a-ae82-4b5cd77c3c8d None admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:54.193779 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c946f-d4b2-420a-ae82-4b5cd77c3c8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:54.194081 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c946f-d4b2-420a-ae82-4b5cd77c3c8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:54.195520 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c946f-d4b2-420a-ae82-4b5cd77c3c8d None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:54.211396 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 633/1259] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:57:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:57:54.219584 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f287fc-9c8a-4cf5-838a-ce60b8ff6cb9 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:54.259758 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:54.359973 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f287fc-9c8a-4cf5-838a-ce60b8ff6cb9 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:54.407585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068e98c-5513-4de9-816f-c889df041657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:54.408491 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068e98c-5513-4de9-816f-c889df041657 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:54.408491 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068e98c-5513-4de9-816f-c889df041657 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:54.408750 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068e98c-5513-4de9-816f-c889df041657 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:54.409634 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1068e98c-5513-4de9-816f-c889df041657 None admin] RBAC: Adding query filter params (role.id=3e1a9ff016c840c5b51ef7e3da9900ac, scope.project.id=36f063d8f65d494a89dbbd72c53417c8, user.id=58380e30789041e4ad9e8652bc628658) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:54.409963 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1068e98c-5513-4de9-816f-c889df041657 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:54.411466 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1068e98c-5513-4de9-816f-c889df041657 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:54.418280 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 627/1260] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:57:54 2022] GET /identity/v3/role_assignments?user.id=58380e30789041e4ad9e8652bc628658&scope.project.id=36f063d8f65d494a89dbbd72c53417c8&role.id=3e1a9ff016c840c5b51ef7e3da9900ac => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:56.566433 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24930b64-3bf9-449c-a577-61ce7e2c4f49 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:56.567257 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24930b64-3bf9-449c-a577-61ce7e2c4f49 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:56.567257 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24930b64-3bf9-449c-a577-61ce7e2c4f49 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:56.568418 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 634/1261] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:57:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:57:56.576901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafe542-9bea-4e35-9e97-181764c6ea23 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:56.577527 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafe542-9bea-4e35-9e97-181764c6ea23 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:56.577609 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafe542-9bea-4e35-9e97-181764c6ea23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:56.599526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-ecafe542-9bea-4e35-9e97-181764c6ea23 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:56.601234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ecafe542-9bea-4e35-9e97-181764c6ea23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:56.681585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 628/1262] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:57:56.708913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24930b64-3bf9-449c-a577-61ce7e2c4f49 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:56.723281 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:56.859767 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24930b64-3bf9-449c-a577-61ce7e2c4f49 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:56.907804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49110aed-de79-457d-9b49-43e9ad0645d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:56.908992 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49110aed-de79-457d-9b49-43e9ad0645d5 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:56.909439 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49110aed-de79-457d-9b49-43e9ad0645d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:56.909789 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49110aed-de79-457d-9b49-43e9ad0645d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:56.926638 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49110aed-de79-457d-9b49-43e9ad0645d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:57:56.931804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-49110aed-de79-457d-9b49-43e9ad0645d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:57:56.931804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:56.931804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:56.931804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:56.931804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:56.931804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:56.931804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:56.931804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:56.931804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:56.931804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:56.931804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:56.931804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:56.931804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:57:56.931804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:57:56.931804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:57:56.931804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:57:56.933245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:57:56.933245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:57:56.933245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:57:56.933245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:57:56.933245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:56.933245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:56.933245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:56.933245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:56.933245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:56.933245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:56.933245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:56.933245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:56.933245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:56.933245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:56.933245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:56.933245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:56.934320 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:56.934320 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:56.934320 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:56.934320 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:56.934320 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:57:56.934320 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:57:56.934320 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:57:56.934320 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:57:56.934320 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:57:56.934320 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:57:56.934320 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:57:56.934320 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:57:56.934320 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 635/1263] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:57:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:56.939825 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecafe542-9bea-4e35-9e97-181764c6ea23 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:56.961320 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:57.061495 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecafe542-9bea-4e35-9e97-181764c6ea23 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:57.109621 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890dbdd7-89fa-49b1-ab7f-180e023cf772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:57.110522 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890dbdd7-89fa-49b1-ab7f-180e023cf772 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:57.110785 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890dbdd7-89fa-49b1-ab7f-180e023cf772 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:57.111249 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890dbdd7-89fa-49b1-ab7f-180e023cf772 None admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:57.112462 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890dbdd7-89fa-49b1-ab7f-180e023cf772 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:57.112931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890dbdd7-89fa-49b1-ab7f-180e023cf772 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:57.114831 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890dbdd7-89fa-49b1-ab7f-180e023cf772 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:57.129147 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 629/1264] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:57:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:57.137350 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49110aed-de79-457d-9b49-43e9ad0645d5 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:57.221617 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:57.262594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49110aed-de79-457d-9b49-43e9ad0645d5 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:57.301718 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed72dd31-0a60-4c08-8872-357be5a23ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:57.302489 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72dd31-0a60-4c08-8872-357be5a23ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:57.302741 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72dd31-0a60-4c08-8872-357be5a23ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:57.303092 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72dd31-0a60-4c08-8872-357be5a23ad8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:57.315628 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed72dd31-0a60-4c08-8872-357be5a23ad8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:57.317299 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed72dd31-0a60-4c08-8872-357be5a23ad8 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:57.335359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-ed72dd31-0a60-4c08-8872-357be5a23ad8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:57.335359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:57.335359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:57.335359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:57.335359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:57.335359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:57.335359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:57.335359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:57.335359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:57.335359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:57.335359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:57.335359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:57.335359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:57:57.335359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:57:57.335359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:57:57.337315 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:57:57.337315 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:57.337315 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:57.337315 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:57.337315 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:57.337315 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:57.337315 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:57.337315 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:57.337315 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:57.337315 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:57.337315 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:57.337315 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:57.337315 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:57.337315 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:57.337315 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:57.337315 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:57.337315 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:57.339398 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:57:57.339398 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:57:57.339398 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:57.339398 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:57:57.339398 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 636/1265] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:57:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:57.343816 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890dbdd7-89fa-49b1-ab7f-180e023cf772 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:57.459488 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:57.487772 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890dbdd7-89fa-49b1-ab7f-180e023cf772 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:57.530178 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1aec13-6309-4941-ab46-6917cbcb24d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:57.530976 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1aec13-6309-4941-ab46-6917cbcb24d2 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:57.531319 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1aec13-6309-4941-ab46-6917cbcb24d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:57.531625 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1aec13-6309-4941-ab46-6917cbcb24d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:57.532436 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1aec13-6309-4941-ab46-6917cbcb24d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:57.532816 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1aec13-6309-4941-ab46-6917cbcb24d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:57.538171 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1aec13-6309-4941-ab46-6917cbcb24d2 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:57.551529 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 630/1266] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:57:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:57.558336 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed72dd31-0a60-4c08-8872-357be5a23ad8 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:57.692076 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed72dd31-0a60-4c08-8872-357be5a23ad8 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:57.728967 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:57.751557 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc49412-1af0-4221-9ef0-d5caf04cf5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:57.752213 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc49412-1af0-4221-9ef0-d5caf04cf5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:57.752623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc49412-1af0-4221-9ef0-d5caf04cf5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:57.752813 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc49412-1af0-4221-9ef0-d5caf04cf5e8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:57.762009 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc49412-1af0-4221-9ef0-d5caf04cf5e8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:57.763671 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc49412-1af0-4221-9ef0-d5caf04cf5e8 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:57.776481 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-bcc49412-1af0-4221-9ef0-d5caf04cf5e8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 13:57:57.776481 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:57.776481 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:57.776481 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:57.776481 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:57.776481 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:57.776481 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:57.776481 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:57.776481 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:57.776481 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:57.776481 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:57.776481 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:57.776481 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:57:57.776481 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:57:57.776481 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:57:57.776481 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:57:57.783075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:57.783075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:57.783075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:57:57.783075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:57:57.783075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:57:57.783075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:57:57.783075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:57.783075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:57.783075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:57.783075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:57.783075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:57.783075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:57.783075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:57.783075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:57.783075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:57.783075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:57.783075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:57.783075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:57.784668 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:57.784668 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:57.784668 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:57:57.784668 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:57:57.784668 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:57:57.784668 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:57:57.784668 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:57:57.784668 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:57:57.784668 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 13:57:57.784668 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:57:57.784668 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 637/1267] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:57:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:57:57.785980 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1aec13-6309-4941-ab46-6917cbcb24d2 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:57.926180 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1aec13-6309-4941-ab46-6917cbcb24d2 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:57.969059 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2df1e5-3fa4-4298-ac03-d372861df310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:57.969680 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2df1e5-3fa4-4298-ac03-d372861df310 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:57.970044 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2df1e5-3fa4-4298-ac03-d372861df310 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:57.970303 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2df1e5-3fa4-4298-ac03-d372861df310 None admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:57.971408 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2df1e5-3fa4-4298-ac03-d372861df310 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:57.971650 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2df1e5-3fa4-4298-ac03-d372861df310 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:57.973296 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2df1e5-3fa4-4298-ac03-d372861df310 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:57.989074 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:58.006054 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 631/1268] 10.222.0.45 () {56 vars in 1143 bytes} [Mon Aug 1 13:57:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:58.013054 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc49412-1af0-4221-9ef0-d5caf04cf5e8 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:58.153736 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc49412-1af0-4221-9ef0-d5caf04cf5e8 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:58.204096 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae59be1-1a15-4caa-810d-b5c67abfdbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:58.204977 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae59be1-1a15-4caa-810d-b5c67abfdbd4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:58.205364 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae59be1-1a15-4caa-810d-b5c67abfdbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:58.205880 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae59be1-1a15-4caa-810d-b5c67abfdbd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:58.222304 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae59be1-1a15-4caa-810d-b5c67abfdbd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:58.223813 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae59be1-1a15-4caa-810d-b5c67abfdbd4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:58.250913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-5ae59be1-1a15-4caa-810d-b5c67abfdbd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:58.250913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:58.250913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:58.250913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:58.250913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:58.250913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:58.250913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:58.250913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:58.250913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:58.250913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:58.250913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:58.250913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:58.250913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:57:58.250913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:57:58.250913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:57:58.252647 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:57:58.252647 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:58.252647 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:58.252647 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:58.252647 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:58.252647 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:58.252647 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:58.252647 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:58.252647 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:58.252647 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:58.252647 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:58.252647 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:58.252647 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:58.252647 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:58.252647 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:58.252647 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:58.252647 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:58.254192 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:57:58.254192 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:57:58.254192 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:58.254192 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:57:58.254192 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 638/1269] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:57:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:58.259761 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2df1e5-3fa4-4298-ac03-d372861df310 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:58.385378 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2df1e5-3fa4-4298-ac03-d372861df310 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:58.432619 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea6bbf-91cf-4357-bb0f-e5c8a2454f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:58.433286 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea6bbf-91cf-4357-bb0f-e5c8a2454f54 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:58.433565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea6bbf-91cf-4357-bb0f-e5c8a2454f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:58.433837 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea6bbf-91cf-4357-bb0f-e5c8a2454f54 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:58.434558 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ea6bbf-91cf-4357-bb0f-e5c8a2454f54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:58.434863 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ea6bbf-91cf-4357-bb0f-e5c8a2454f54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:58.436343 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ea6bbf-91cf-4357-bb0f-e5c8a2454f54 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:58.450311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 632/1270] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:57:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:58.457843 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae59be1-1a15-4caa-810d-b5c67abfdbd4 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:58.499051 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:58.602300 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae59be1-1a15-4caa-810d-b5c67abfdbd4 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:58.644219 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61a996-59ad-4bdb-8f40-916b52270d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:58.645346 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61a996-59ad-4bdb-8f40-916b52270d53 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:58.645848 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61a996-59ad-4bdb-8f40-916b52270d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:58.646349 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61a996-59ad-4bdb-8f40-916b52270d53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:58.658640 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61a996-59ad-4bdb-8f40-916b52270d53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:58.661168 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61a996-59ad-4bdb-8f40-916b52270d53 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:58.679642 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-3b61a996-59ad-4bdb-8f40-916b52270d53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:57:58.679642 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:58.679642 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:58.679642 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:58.679642 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:58.679642 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:58.679642 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:58.679642 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:58.679642 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:58.679642 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:58.679642 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:58.679642 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:58.679642 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:57:58.679642 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:57:58.679642 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:57:58.680777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:57:58.680777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:58.680777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:58.680777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:58.680777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:58.680777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:58.680777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:58.680777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:58.680777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:58.680777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:58.680777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:58.680777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:58.680777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:58.680777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:58.680777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:58.680777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:58.680777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:58.682013 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:57:58.682013 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:57:58.682013 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:57:58.682013 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:57:58.682013 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:57:58.682013 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:57:58.682013 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:57:58.682013 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:57:58.682013 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 639/1271] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:57:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:58.690262 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea6bbf-91cf-4357-bb0f-e5c8a2454f54 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:58.766417 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:58.845717 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea6bbf-91cf-4357-bb0f-e5c8a2454f54 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:58.887561 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f087508-fbd0-4d30-b6a5-b97ad7894408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:58.888195 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f087508-fbd0-4d30-b6a5-b97ad7894408 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:58.888499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f087508-fbd0-4d30-b6a5-b97ad7894408 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:58.888771 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f087508-fbd0-4d30-b6a5-b97ad7894408 None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:58.889557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f087508-fbd0-4d30-b6a5-b97ad7894408 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:58.889868 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f087508-fbd0-4d30-b6a5-b97ad7894408 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:58.891367 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f087508-fbd0-4d30-b6a5-b97ad7894408 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:58.906443 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 633/1272] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:57:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:58.913257 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61a996-59ad-4bdb-8f40-916b52270d53 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:59.030876 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:59.042471 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61a996-59ad-4bdb-8f40-916b52270d53 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:59.081951 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f90eb39-2905-4540-840f-0f9f5a06e954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:59.082678 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90eb39-2905-4540-840f-0f9f5a06e954 None admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:59.082960 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90eb39-2905-4540-840f-0f9f5a06e954 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:59.083258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90eb39-2905-4540-840f-0f9f5a06e954 None admin] PATH_INFO: `/v3/projects/36f063d8f65d494a89dbbd72c53417c8/users/58380e30789041e4ad9e8652bc628658/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:59.085394 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f90eb39-2905-4540-840f-0f9f5a06e954 None admin] RBAC: Authorizing `identity:create_grant(project_id=36f063d8f65d494a89dbbd72c53417c8, user_id=58380e30789041e4ad9e8652bc628658, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:59.086899 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f90eb39-2905-4540-840f-0f9f5a06e954 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:59.108403 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 640/1273] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 13:57:58 2022] PUT /identity/v3/projects/36f063d8f65d494a89dbbd72c53417c8/users/58380e30789041e4ad9e8652bc628658/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:58:01.125732 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e476d-f249-4656-9f20-e72d3b4c6c4b None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:01.125732 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e476d-f249-4656-9f20-e72d3b4c6c4b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:01.125732 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e476d-f249-4656-9f20-e72d3b4c6c4b None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:01.127421 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 634/1274] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:58:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:58:01.133816 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93409776-e413-4db5-b39e-7ec8ae4afda0 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:01.134390 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93409776-e413-4db5-b39e-7ec8ae4afda0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:01.134390 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93409776-e413-4db5-b39e-7ec8ae4afda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:01.157229 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-93409776-e413-4db5-b39e-7ec8ae4afda0 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:58:01.158466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-93409776-e413-4db5-b39e-7ec8ae4afda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:58:01.230392 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 641/1275] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:58:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:58:01.254079 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e476d-f249-4656-9f20-e72d3b4c6c4b None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:01.370049 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e476d-f249-4656-9f20-e72d3b4c6c4b None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:01.390052 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:01.410854 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77708d-543b-4f30-b41d-6bd8226dff62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:01.411509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77708d-543b-4f30-b41d-6bd8226dff62 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:01.411795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77708d-543b-4f30-b41d-6bd8226dff62 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:01.412065 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77708d-543b-4f30-b41d-6bd8226dff62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:01.424890 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe77708d-543b-4f30-b41d-6bd8226dff62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:58:01.430426 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-fe77708d-543b-4f30-b41d-6bd8226dff62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:58:01.430426 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:58:01.430426 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:58:01.430426 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:58:01.430426 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:58:01.430426 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:58:01.430426 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:58:01.430426 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:58:01.430426 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:58:01.430426 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:58:01.430426 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:58:01.430426 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:58:01.430426 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:58:01.430426 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:58:01.430426 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:58:01.430426 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:58:01.431591 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:58:01.431591 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:58:01.431591 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:58:01.431591 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:58:01.431591 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:58:01.431591 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:58:01.431591 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:58:01.431591 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:58:01.431591 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:58:01.431591 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:58:01.431591 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:58:01.431591 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:58:01.431591 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:58:01.431591 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:58:01.431591 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:58:01.431591 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:58:01.437457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:58:01.437457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:58:01.437457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:58:01.437457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:58:01.437457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:58:01.437457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:58:01.437457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:58:01.437457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:58:01.437457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:58:01.437457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:58:01.437457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:58:01.437457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:58:01.437457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 635/1276] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 13:58:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:58:01.438419 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93409776-e413-4db5-b39e-7ec8ae4afda0 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:01.530183 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:01.559139 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93409776-e413-4db5-b39e-7ec8ae4afda0 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:01.599987 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812dcd3e-ea86-4ecc-bd97-f67869aeee18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:01.601136 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812dcd3e-ea86-4ecc-bd97-f67869aeee18 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:01.601136 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812dcd3e-ea86-4ecc-bd97-f67869aeee18 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:01.601356 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812dcd3e-ea86-4ecc-bd97-f67869aeee18 None admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:01.601896 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812dcd3e-ea86-4ecc-bd97-f67869aeee18 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:58:01.602756 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812dcd3e-ea86-4ecc-bd97-f67869aeee18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:01.603604 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812dcd3e-ea86-4ecc-bd97-f67869aeee18 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:01.613221 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 642/1277] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 13:58:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:58:01.619592 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77708d-543b-4f30-b41d-6bd8226dff62 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:01.731816 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77708d-543b-4f30-b41d-6bd8226dff62 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:01.769444 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf3d45-b4ea-4de5-ab43-ae61094b8bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:01.770091 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf3d45-b4ea-4de5-ab43-ae61094b8bef None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:01.770385 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf3d45-b4ea-4de5-ab43-ae61094b8bef None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:01.770658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf3d45-b4ea-4de5-ab43-ae61094b8bef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:01.783016 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf3d45-b4ea-4de5-ab43-ae61094b8bef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:01.784620 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf3d45-b4ea-4de5-ab43-ae61094b8bef None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:01.802324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-abbf3d45-b4ea-4de5-ab43-ae61094b8bef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:58:01.802324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:58:01.802324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:58:01.802324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:58:01.802324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:58:01.802324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:58:01.802324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:58:01.802324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:58:01.802324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:58:01.802324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:58:01.802324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:58:01.802324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:58:01.802324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:58:01.802324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:58:01.802324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:58:01.803577 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:58:01.803577 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:58:01.803577 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:58:01.803577 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:58:01.803577 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:58:01.803577 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:58:01.803577 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:58:01.803577 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:58:01.803577 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:58:01.803577 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:58:01.803577 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:58:01.803577 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:58:01.803577 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:58:01.803577 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:58:01.803577 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:58:01.803577 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:58:01.803577 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:58:01.804949 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:58:01.804949 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:58:01.804949 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:58:01.804949 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:58:01.804949 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 636/1278] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:58:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:58:01.809630 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812dcd3e-ea86-4ecc-bd97-f67869aeee18 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:01.947801 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812dcd3e-ea86-4ecc-bd97-f67869aeee18 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:01.985317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750baaf-53c7-4ef8-a829-e5c9ab0f6b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:01.985854 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750baaf-53c7-4ef8-a829-e5c9ab0f6b44 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:01.986138 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750baaf-53c7-4ef8-a829-e5c9ab0f6b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:01.986415 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750baaf-53c7-4ef8-a829-e5c9ab0f6b44 None admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:01.987185 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d750baaf-53c7-4ef8-a829-e5c9ab0f6b44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:58:01.987476 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d750baaf-53c7-4ef8-a829-e5c9ab0f6b44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:01.988898 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d750baaf-53c7-4ef8-a829-e5c9ab0f6b44 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:02.001370 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 643/1279] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:58:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:58:02.009905 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf3d45-b4ea-4de5-ab43-ae61094b8bef None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:02.047308 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:02.136969 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf3d45-b4ea-4de5-ab43-ae61094b8bef None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:02.178066 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5256f-b6ba-45f6-888f-59bf0e732ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:02.178949 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5256f-b6ba-45f6-888f-59bf0e732ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:02.179321 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5256f-b6ba-45f6-888f-59bf0e732ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:02.179688 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5256f-b6ba-45f6-888f-59bf0e732ad4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:02.188807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df5256f-b6ba-45f6-888f-59bf0e732ad4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:02.190261 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df5256f-b6ba-45f6-888f-59bf0e732ad4 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:02.202653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-7df5256f-b6ba-45f6-888f-59bf0e732ad4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 13:58:02.202653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:58:02.202653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:58:02.202653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:58:02.202653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:58:02.202653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:58:02.202653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:58:02.202653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:58:02.202653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:58:02.202653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:58:02.202653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:58:02.202653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:58:02.202653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:58:02.202653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:58:02.202653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:58:02.202653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:58:02.203876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:58:02.203876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:58:02.203876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:58:02.203876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:58:02.203876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:58:02.203876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:58:02.203876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:58:02.203876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:58:02.203876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:58:02.203876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:58:02.203876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:58:02.203876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:58:02.203876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:58:02.203876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:58:02.203876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:58:02.203876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:58:02.203876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:58:02.203876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:58:02.205076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:58:02.205076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:58:02.205076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:58:02.205076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:58:02.205076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:58:02.205076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:58:02.205076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:58:02.205076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:58:02.205076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 13:58:02.205076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:58:02.205076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 637/1280] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:58:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:58:02.210180 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750baaf-53c7-4ef8-a829-e5c9ab0f6b44 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:02.214334 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:02.320885 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750baaf-53c7-4ef8-a829-e5c9ab0f6b44 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:02.358146 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da9cf9-1f52-42d4-9d9d-5ed924d1c08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:02.358771 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da9cf9-1f52-42d4-9d9d-5ed924d1c08a None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:02.359054 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da9cf9-1f52-42d4-9d9d-5ed924d1c08a None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:02.359317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da9cf9-1f52-42d4-9d9d-5ed924d1c08a None admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:02.360119 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da9cf9-1f52-42d4-9d9d-5ed924d1c08a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:58:02.360456 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da9cf9-1f52-42d4-9d9d-5ed924d1c08a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:02.361901 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da9cf9-1f52-42d4-9d9d-5ed924d1c08a None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:02.381990 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 644/1281] 10.222.0.45 () {56 vars in 1143 bytes} [Mon Aug 1 13:58:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:58:02.388212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5256f-b6ba-45f6-888f-59bf0e732ad4 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:02.489929 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:02.502541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5256f-b6ba-45f6-888f-59bf0e732ad4 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:02.537442 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2032a6-5abb-4bd8-84eb-8e180efb55a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:02.538148 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2032a6-5abb-4bd8-84eb-8e180efb55a0 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:02.538425 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2032a6-5abb-4bd8-84eb-8e180efb55a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:02.538693 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2032a6-5abb-4bd8-84eb-8e180efb55a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:02.611466 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2032a6-5abb-4bd8-84eb-8e180efb55a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:02.612902 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2032a6-5abb-4bd8-84eb-8e180efb55a0 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:02.721205 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:02.733412 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 638/1282] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 13:58:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:58:02.739879 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da9cf9-1f52-42d4-9d9d-5ed924d1c08a None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:02.807860 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:02.850921 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da9cf9-1f52-42d4-9d9d-5ed924d1c08a None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:02.888093 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b721ff-a69b-4d58-9dad-891425df1a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:02.888669 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b721ff-a69b-4d58-9dad-891425df1a27 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:02.888931 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b721ff-a69b-4d58-9dad-891425df1a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:02.889223 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b721ff-a69b-4d58-9dad-891425df1a27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:02.900320 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b721ff-a69b-4d58-9dad-891425df1a27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:02.901872 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b721ff-a69b-4d58-9dad-891425df1a27 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:02.918577 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-d4b721ff-a69b-4d58-9dad-891425df1a27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:58:02.918577 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:58:02.918577 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:58:02.918577 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:58:02.918577 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:58:02.918577 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:58:02.918577 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:58:02.918577 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:58:02.918577 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:58:02.918577 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:58:02.918577 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:58:02.918577 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:58:02.918577 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:58:02.918577 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:58:02.918577 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:58:02.920165 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:58:02.920165 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:58:02.920165 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:58:02.920165 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:58:02.920165 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:58:02.920165 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:58:02.920165 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:58:02.920165 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:58:02.920165 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:58:02.920165 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:58:02.920165 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:58:02.920165 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:58:02.920165 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:58:02.920165 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:58:02.920165 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:58:02.920165 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:58:02.920165 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:58:02.922009 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:58:02.922009 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:58:02.922009 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:58:02.922009 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:58:02.922009 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 645/1283] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 13:58:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:58:02.927076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2032a6-5abb-4bd8-84eb-8e180efb55a0 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:03.030461 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2032a6-5abb-4bd8-84eb-8e180efb55a0 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:03.068504 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d90b15-afd5-4890-92d7-bddf8d260008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:03.069162 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d90b15-afd5-4890-92d7-bddf8d260008 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:03.069433 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d90b15-afd5-4890-92d7-bddf8d260008 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:03.069697 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d90b15-afd5-4890-92d7-bddf8d260008 None admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:03.070386 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d90b15-afd5-4890-92d7-bddf8d260008 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:58:03.070679 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d90b15-afd5-4890-92d7-bddf8d260008 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:03.072058 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d90b15-afd5-4890-92d7-bddf8d260008 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:03.084541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 639/1284] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 13:58:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:58:03.090484 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b721ff-a69b-4d58-9dad-891425df1a27 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:03.217364 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b721ff-a69b-4d58-9dad-891425df1a27 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:03.231766 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:03.261320 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9d016-3c08-4461-a66f-64c74260e5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:03.261970 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9d016-3c08-4461-a66f-64c74260e5a3 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:03.262249 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9d016-3c08-4461-a66f-64c74260e5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:03.262505 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9d016-3c08-4461-a66f-64c74260e5a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:03.270459 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d9d016-3c08-4461-a66f-64c74260e5a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:03.271964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d9d016-3c08-4461-a66f-64c74260e5a3 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:03.284732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application [None req-a2d9d016-3c08-4461-a66f-64c74260e5a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:58:03.284732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:58:03.284732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:58:03.284732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:58:03.284732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:58:03.284732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:58:03.284732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:58:03.284732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:58:03.284732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:58:03.284732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:58:03.284732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:58:03.284732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:58:03.284732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:58:03.284732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:58:03.284732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:58:03.286029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:58:03.286029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:58:03.286029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:58:03.286029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:58:03.286029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:58:03.286029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:58:03.286029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:58:03.286029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:58:03.286029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application with Lock( Aug 01 13:58:03.286029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:58:03.286029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:58:03.286029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:58:03.286029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:58:03.286029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:58:03.286029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:58:03.286029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:58:03.286029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:58:03.287200 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:58:03.287200 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:58:03.287200 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:58:03.287200 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:58:03.287200 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:58:03.287200 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:58:03.287200 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:58:03.287200 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: ERROR keystone.server.flask.application Aug 01 13:58:03.287200 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 646/1285] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 13:58:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:58:03.292254 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d90b15-afd5-4890-92d7-bddf8d260008 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:03.346584 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:03.409595 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d90b15-afd5-4890-92d7-bddf8d260008 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:03.455526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47969e3-fe08-4a19-9f02-3ffbeca3fe43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:03.456148 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47969e3-fe08-4a19-9f02-3ffbeca3fe43 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:03.456457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47969e3-fe08-4a19-9f02-3ffbeca3fe43 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:03.456724 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47969e3-fe08-4a19-9f02-3ffbeca3fe43 None admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:03.457554 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47969e3-fe08-4a19-9f02-3ffbeca3fe43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:58:03.457854 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47969e3-fe08-4a19-9f02-3ffbeca3fe43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:03.459293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47969e3-fe08-4a19-9f02-3ffbeca3fe43 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:03.472405 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 640/1286] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 13:58:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:58:03.478711 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9d016-3c08-4461-a66f-64c74260e5a3 None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:03.584129 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9d016-3c08-4461-a66f-64c74260e5a3 None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:03.619279 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e5444-f01d-427c-a4b5-5ff71c02abc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:03.619900 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e5444-f01d-427c-a4b5-5ff71c02abc4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:03.620177 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e5444-f01d-427c-a4b5-5ff71c02abc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:03.620473 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e5444-f01d-427c-a4b5-5ff71c02abc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:03.621262 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e5444-f01d-427c-a4b5-5ff71c02abc4 None admin] RBAC: Adding query filter params (role.id=3e1a9ff016c840c5b51ef7e3da9900ac, scope.project.id=36f063d8f65d494a89dbbd72c53417c8, user.id=58380e30789041e4ad9e8652bc628658) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:58:03.621560 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e5444-f01d-427c-a4b5-5ff71c02abc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:03.622986 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e5444-f01d-427c-a4b5-5ff71c02abc4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:03.629613 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 647/1287] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 13:58:03 2022] GET /identity/v3/role_assignments?user.id=58380e30789041e4ad9e8652bc628658&scope.project.id=36f063d8f65d494a89dbbd72c53417c8&role.id=3e1a9ff016c840c5b51ef7e3da9900ac => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:58:05.653956 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b62a71e-fd75-4d39-bc9f-519507cf7f6c None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:05.654599 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b62a71e-fd75-4d39-bc9f-519507cf7f6c None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:05.654599 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b62a71e-fd75-4d39-bc9f-519507cf7f6c None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:05.655872 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 641/1288] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:58:05.661332 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09537e24-73ba-47a2-9352-af8d7a21da95 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:05.661772 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09537e24-73ba-47a2-9352-af8d7a21da95 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:05.661881 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09537e24-73ba-47a2-9352-af8d7a21da95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:05.680881 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-09537e24-73ba-47a2-9352-af8d7a21da95 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:58:05.682075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-09537e24-73ba-47a2-9352-af8d7a21da95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:58:05.714488 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:05.761582 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 648/1289] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:58:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:58:05.784408 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b62a71e-fd75-4d39-bc9f-519507cf7f6c None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:05.904620 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b62a71e-fd75-4d39-bc9f-519507cf7f6c None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:05.915923 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:05.940057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63fe2a-d24a-4e39-aedd-97d0d357339f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:05.940739 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63fe2a-d24a-4e39-aedd-97d0d357339f None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:05.941015 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63fe2a-d24a-4e39-aedd-97d0d357339f None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:05.941274 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63fe2a-d24a-4e39-aedd-97d0d357339f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:05.941909 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63fe2a-d24a-4e39-aedd-97d0d357339f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:05.943274 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63fe2a-d24a-4e39-aedd-97d0d357339f None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:05.952852 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-8e63fe2a-d24a-4e39-aedd-97d0d357339f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 13:58:05.952852 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:58:05.952852 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:58:05.952852 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:58:05.952852 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:58:05.952852 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:58:05.952852 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:58:05.952852 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:58:05.952852 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 13:58:05.952852 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:58:05.952852 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:58:05.952852 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:58:05.952852 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:58:05.952852 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:58:05.952852 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:58:05.952852 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:58:05.952852 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:58:05.954084 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:58:05.954084 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:58:05.954084 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:58:05.954084 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:58:05.954084 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:58:05.954084 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:58:05.954084 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:58:05.954084 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:58:05.954084 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:58:05.954084 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:58:05.954084 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:58:05.954084 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:58:05.954084 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:58:05.954084 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:58:05.954084 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:58:05.954084 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:58:05.955224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:58:05.955224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:58:05.955224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:58:05.955224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:58:05.955224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:58:05.955224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:58:05.955224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:58:05.955224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:58:05.955224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:58:05.955224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:58:05.955224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:58:05.955224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:58:05.955224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 13:58:05.955224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:58:05.955224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 642/1290] 10.222.0.45 () {56 vars in 1117 bytes} [Mon Aug 1 13:58:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:58:05.960798 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09537e24-73ba-47a2-9352-af8d7a21da95 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:06.076553 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09537e24-73ba-47a2-9352-af8d7a21da95 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:06.128398 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbec6e-b63f-423c-b427-27553e37f1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:06.129436 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbec6e-b63f-423c-b427-27553e37f1d4 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:06.129871 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbec6e-b63f-423c-b427-27553e37f1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:06.130246 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbec6e-b63f-423c-b427-27553e37f1d4 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:06.131148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fbec6e-b63f-423c-b427-27553e37f1d4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:58:06.131570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fbec6e-b63f-423c-b427-27553e37f1d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:06.133316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fbec6e-b63f-423c-b427-27553e37f1d4 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:06.139634 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 649/1291] 10.222.0.45 () {56 vars in 1116 bytes} [Mon Aug 1 13:58:05 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:58:06.146450 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63fe2a-d24a-4e39-aedd-97d0d357339f None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:06.264685 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63fe2a-d24a-4e39-aedd-97d0d357339f None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:06.300202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf8d4a-9abd-483c-b4dc-1ac7b93cc646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:06.300872 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf8d4a-9abd-483c-b4dc-1ac7b93cc646 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:06.301153 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf8d4a-9abd-483c-b4dc-1ac7b93cc646 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:06.301428 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf8d4a-9abd-483c-b4dc-1ac7b93cc646 None admin] PATH_INFO: `/v3/services` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:06.302149 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf8d4a-9abd-483c-b4dc-1ac7b93cc646 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:58:06.302453 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf8d4a-9abd-483c-b4dc-1ac7b93cc646 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:06.303905 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf8d4a-9abd-483c-b4dc-1ac7b93cc646 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:06.310388 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 643/1292] 10.222.0.45 () {56 vars in 1116 bytes} [Mon Aug 1 13:58:06 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:58:08.330546 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326d811-3fd3-446f-8730-2f15cd364b5f None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:08.331160 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326d811-3fd3-446f-8730-2f15cd364b5f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:08.331160 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326d811-3fd3-446f-8730-2f15cd364b5f None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:08.332540 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 650/1293] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:58:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:58:08.338014 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911129aa-543e-4faa-b8db-185f418f6235 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:08.338481 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911129aa-543e-4faa-b8db-185f418f6235 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:08.338595 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911129aa-543e-4faa-b8db-185f418f6235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:08.358566 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-911129aa-543e-4faa-b8db-185f418f6235 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:58:08.359804 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-911129aa-543e-4faa-b8db-185f418f6235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:58:08.392569 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:08.436519 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 644/1294] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:58:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:58:08.459206 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326d811-3fd3-446f-8730-2f15cd364b5f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:08.492724 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:08.569574 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326d811-3fd3-446f-8730-2f15cd364b5f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:08.605235 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f3dfe8-37b3-4321-bd46-5451457aa1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:08.605766 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3dfe8-37b3-4321-bd46-5451457aa1ff None admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:08.606056 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3dfe8-37b3-4321-bd46-5451457aa1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:08.606325 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3dfe8-37b3-4321-bd46-5451457aa1ff None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:08.607019 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f3dfe8-37b3-4321-bd46-5451457aa1ff None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:08.608435 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f3dfe8-37b3-4321-bd46-5451457aa1ff None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:08.625018 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 651/1295] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Aug 1 13:58:08 2022] POST /identity/v3/services => generated 246 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:58:10.664847 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12886a6d-2c78-421a-933d-45da5402a2bf None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:10.665458 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12886a6d-2c78-421a-933d-45da5402a2bf None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:10.665458 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12886a6d-2c78-421a-933d-45da5402a2bf None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:10.666758 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 645/1296] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:58:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:58:10.673057 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfccde0-0803-4b9b-a407-b92d3d2af17e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:10.673648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfccde0-0803-4b9b-a407-b92d3d2af17e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:10.673755 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfccde0-0803-4b9b-a407-b92d3d2af17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:10.694023 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-5bfccde0-0803-4b9b-a407-b92d3d2af17e None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:58:10.695250 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5bfccde0-0803-4b9b-a407-b92d3d2af17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:58:10.774462 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 652/1297] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:58:10 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:58:10.799109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12886a6d-2c78-421a-933d-45da5402a2bf None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:10.913283 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12886a6d-2c78-421a-933d-45da5402a2bf None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:10.951603 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7d743a-d11b-40c2-83fc-fe97444694b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:10.952322 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d743a-d11b-40c2-83fc-fe97444694b5 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:10.952615 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d743a-d11b-40c2-83fc-fe97444694b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:10.952885 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d743a-d11b-40c2-83fc-fe97444694b5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:10.953529 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7d743a-d11b-40c2-83fc-fe97444694b5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:10.954975 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7d743a-d11b-40c2-83fc-fe97444694b5 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:10.965384 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-cf7d743a-d11b-40c2-83fc-fe97444694b5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 13:58:10.965384 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:58:10.965384 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:58:10.965384 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:58:10.965384 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:58:10.965384 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:58:10.965384 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:58:10.965384 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:58:10.965384 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 13:58:10.965384 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:58:10.965384 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:58:10.965384 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:58:10.965384 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:58:10.965384 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:58:10.965384 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:58:10.965384 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:58:10.965384 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:58:10.966762 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:58:10.966762 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:58:10.966762 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:58:10.966762 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:58:10.966762 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:58:10.966762 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:58:10.966762 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:58:10.966762 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:58:10.966762 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:58:10.966762 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:58:10.966762 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:58:10.966762 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:58:10.966762 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:58:10.966762 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:58:10.966762 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:58:10.966762 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:58:10.967668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:58:10.967668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:58:10.967668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:58:10.967668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:58:10.967668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:58:10.967668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:58:10.967668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:58:10.967668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:58:10.967668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:58:10.967668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:58:10.967668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:58:10.967668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:58:10.967668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 13:58:10.967668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:58:10.967668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 646/1298] 10.222.0.45 () {56 vars in 1117 bytes} [Mon Aug 1 13:58:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:58:10.973716 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfccde0-0803-4b9b-a407-b92d3d2af17e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:10.996118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:11.095861 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfccde0-0803-4b9b-a407-b92d3d2af17e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:11.143903 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7be211-1f1e-4d90-afe6-89967bc8ca2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:11.144819 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7be211-1f1e-4d90-afe6-89967bc8ca2c None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:11.145098 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7be211-1f1e-4d90-afe6-89967bc8ca2c None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:11.145366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7be211-1f1e-4d90-afe6-89967bc8ca2c None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:11.146054 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7be211-1f1e-4d90-afe6-89967bc8ca2c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:58:11.146357 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7be211-1f1e-4d90-afe6-89967bc8ca2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:11.147818 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7be211-1f1e-4d90-afe6-89967bc8ca2c None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:11.154476 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 653/1299] 10.222.0.45 () {56 vars in 1116 bytes} [Mon Aug 1 13:58:10 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:58:11.161302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7d743a-d11b-40c2-83fc-fe97444694b5 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:11.204238 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:11.281077 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7d743a-d11b-40c2-83fc-fe97444694b5 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:11.318276 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5079fc52-15e9-4ab2-a516-bc53b39618b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:11.318913 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079fc52-15e9-4ab2-a516-bc53b39618b1 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:11.319166 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079fc52-15e9-4ab2-a516-bc53b39618b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:11.319431 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079fc52-15e9-4ab2-a516-bc53b39618b1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:11.320220 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5079fc52-15e9-4ab2-a516-bc53b39618b1 None admin] RBAC: Adding query filter params (interface=public, service_id=13a8d18a29f64cab935ea96a707a100e, region_id=RegionOne) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:58:11.320564 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5079fc52-15e9-4ab2-a516-bc53b39618b1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:11.321951 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5079fc52-15e9-4ab2-a516-bc53b39618b1 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:11.328210 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 647/1300] 10.222.0.45 () {56 vars in 1251 bytes} [Mon Aug 1 13:58:11 2022] GET /identity/v3/endpoints?service_id=13a8d18a29f64cab935ea96a707a100e&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:58:11.337185 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7be211-1f1e-4d90-afe6-89967bc8ca2c None admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:11.430897 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:11.454470 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7be211-1f1e-4d90-afe6-89967bc8ca2c None admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:11.497917 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16674a12-308a-45a9-9d0c-273bbbef25a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:11.498541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16674a12-308a-45a9-9d0c-273bbbef25a0 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:11.498817 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16674a12-308a-45a9-9d0c-273bbbef25a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:11.499090 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16674a12-308a-45a9-9d0c-273bbbef25a0 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:11.499728 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16674a12-308a-45a9-9d0c-273bbbef25a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:11.501151 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16674a12-308a-45a9-9d0c-273bbbef25a0 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:11.508659 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 654/1301] 10.222.0.45 () {56 vars in 1087 bytes} [Mon Aug 1 13:58:11 2022] GET /identity/v3/services => generated 2191 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 13:58:13.514180 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92298ffb-f7cf-42a4-92e9-1b8a48539beb None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:13.514943 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92298ffb-f7cf-42a4-92e9-1b8a48539beb None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:13.514943 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92298ffb-f7cf-42a4-92e9-1b8a48539beb None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:13.517258 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 648/1302] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:58:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:58:13.525355 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3b451-165d-4504-b859-46c62c974a7b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:13.525785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3b451-165d-4504-b859-46c62c974a7b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:13.525925 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3b451-165d-4504-b859-46c62c974a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:13.554578 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a0d3b451-165d-4504-b859-46c62c974a7b None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:58:13.555942 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a0d3b451-165d-4504-b859-46c62c974a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:58:13.630235 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 655/1303] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:58:13 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:58:13.653998 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92298ffb-f7cf-42a4-92e9-1b8a48539beb None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:13.767444 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:13.774645 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92298ffb-f7cf-42a4-92e9-1b8a48539beb None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:13.813252 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906221a9-9bdf-457a-b1b5-bab27846ab76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:13.813956 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906221a9-9bdf-457a-b1b5-bab27846ab76 None admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:13.814210 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906221a9-9bdf-457a-b1b5-bab27846ab76 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:13.814476 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906221a9-9bdf-457a-b1b5-bab27846ab76 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:13.815109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906221a9-9bdf-457a-b1b5-bab27846ab76 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:13.816591 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906221a9-9bdf-457a-b1b5-bab27846ab76 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:13.830310 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application [None req-906221a9-9bdf-457a-b1b5-bab27846ab76 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 13:58:13.830310 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:58:13.830310 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:58:13.830310 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:58:13.830310 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:58:13.830310 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:58:13.830310 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:58:13.830310 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:58:13.830310 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 13:58:13.830310 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:58:13.830310 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:58:13.830310 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:58:13.830310 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:58:13.830310 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:58:13.830310 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:58:13.830310 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:58:13.830310 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:58:13.831556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:58:13.831556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:58:13.831556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:58:13.831556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:58:13.831556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:58:13.831556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:58:13.831556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:58:13.831556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:58:13.831556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:58:13.831556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:58:13.831556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:58:13.831556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:58:13.831556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:58:13.831556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:58:13.831556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:58:13.831556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:58:13.832725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:58:13.832725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application with Lock( Aug 01 13:58:13.832725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:58:13.832725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:58:13.832725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:58:13.832725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:58:13.832725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:58:13.832725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:58:13.832725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:58:13.832725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:58:13.832725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:58:13.832725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:58:13.832725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 13:58:13.832725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: ERROR keystone.server.flask.application Aug 01 13:58:13.832725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 649/1304] 10.222.0.45 () {56 vars in 1117 bytes} [Mon Aug 1 13:58:13 2022] GET /identity/v3/services/placement => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:58:13.839520 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d3b451-165d-4504-b859-46c62c974a7b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:13.924698 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:13.952244 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d3b451-165d-4504-b859-46c62c974a7b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:13.988175 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02ed25-430c-41fd-b77b-f52980f13100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:13.988932 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02ed25-430c-41fd-b77b-f52980f13100 None admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:13.989212 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02ed25-430c-41fd-b77b-f52980f13100 None admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:13.989486 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02ed25-430c-41fd-b77b-f52980f13100 None admin] PATH_INFO: `/v3/services` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:13.990193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b02ed25-430c-41fd-b77b-f52980f13100 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:58:13.990496 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b02ed25-430c-41fd-b77b-f52980f13100 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:13.991891 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b02ed25-430c-41fd-b77b-f52980f13100 None admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:13.997330 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 656/1305] 10.222.0.45 () {56 vars in 1116 bytes} [Mon Aug 1 13:58:13 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:58:14.008961 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906221a9-9bdf-457a-b1b5-bab27846ab76 None admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:14.127790 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906221a9-9bdf-457a-b1b5-bab27846ab76 None admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:14.180190 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64273315-45d1-4799-b059-00780548f963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:14.180571 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64273315-45d1-4799-b059-00780548f963 None admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:14.180794 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64273315-45d1-4799-b059-00780548f963 None admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:14.181317 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64273315-45d1-4799-b059-00780548f963 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:14.182823 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64273315-45d1-4799-b059-00780548f963 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:14.184799 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64273315-45d1-4799-b059-00780548f963 None admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:14.209729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 650/1306] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 13:58:14 2022] POST /identity/v3/endpoints => generated 331 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:58:23.994354 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=30 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:24.206479 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:27.851491 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8004b77-084d-4c31-bbaf-3873ae5ffa8d None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:27.852615 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8004b77-084d-4c31-bbaf-3873ae5ffa8d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:27.853136 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8004b77-084d-4c31-bbaf-3873ae5ffa8d None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:27.858869 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 657/1307] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:59:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:59:27.861023 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46dd20-8224-4b79-961f-f848e1c59dcb None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:27.862112 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46dd20-8224-4b79-961f-f848e1c59dcb None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:27.862627 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46dd20-8224-4b79-961f-f848e1c59dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:28.087871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-ad46dd20-8224-4b79-961f-f848e1c59dcb None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:59:28.089131 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ad46dd20-8224-4b79-961f-f848e1c59dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:28.276328 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 651/1308] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:59:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 13:59:28.329456 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75289f87-0cb5-4379-a40a-731254343b77 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:28.330064 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75289f87-0cb5-4379-a40a-731254343b77 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:28.330338 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75289f87-0cb5-4379-a40a-731254343b77 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:28.331783 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 658/1309] 10.222.0.45 () {54 vars in 965 bytes} [Mon Aug 1 13:59:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:59:28.338432 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d58474b-59a0-49bf-9297-93a531ac7010 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:28.338739 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d58474b-59a0-49bf-9297-93a531ac7010 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:28.338988 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d58474b-59a0-49bf-9297-93a531ac7010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:28.452744 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-3d58474b-59a0-49bf-9297-93a531ac7010 None None] MFA Rules not processed for user `6dd44b7a78594288b6aabb393fd6324a`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:59:28.454128 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3d58474b-59a0-49bf-9297-93a531ac7010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:28.485351 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:28.561365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 652/1310] 10.222.0.45 () {58 vars in 1062 bytes} [Mon Aug 1 13:59:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 13:59:28.570848 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75289f87-0cb5-4379-a40a-731254343b77 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:28.572770 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-75289f87-0cb5-4379-a40a-731254343b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:28.697022 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75289f87-0cb5-4379-a40a-731254343b77 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:28.792484 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea5998-d21e-4269-8236-269f49c5e03d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:28.793574 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea5998-d21e-4269-8236-269f49c5e03d service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:28.794128 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea5998-d21e-4269-8236-269f49c5e03d service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:28.794592 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea5998-d21e-4269-8236-269f49c5e03d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:28.796511 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f9ea5998-d21e-4269-8236-269f49c5e03d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:28.865151 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:28.934360 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea5998-d21e-4269-8236-269f49c5e03d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:28.935913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea5998-d21e-4269-8236-269f49c5e03d service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:28.964223 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 659/1311] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 13:59:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 13:59:31.409370 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ada64e-f023-4957-b0d6-e34a1a843fde None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:31.409972 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ada64e-f023-4957-b0d6-e34a1a843fde None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:31.410273 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ada64e-f023-4957-b0d6-e34a1a843fde None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:31.412496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 653/1312] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:59:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:59:31.421621 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7098b-3086-4a6e-80ca-b844c5bea56f None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:31.422209 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7098b-3086-4a6e-80ca-b844c5bea56f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:31.422209 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7098b-3086-4a6e-80ca-b844c5bea56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:31.531054 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-e0b7098b-3086-4a6e-80ca-b844c5bea56f None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:59:31.533121 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e0b7098b-3086-4a6e-80ca-b844c5bea56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:31.658053 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 660/1313] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:59:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 13:59:31.719456 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc4122-8e99-4bf1-8869-ea3eca1428ed None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:31.719809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc4122-8e99-4bf1-8869-ea3eca1428ed None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:31.720088 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc4122-8e99-4bf1-8869-ea3eca1428ed None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:31.721593 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 654/1314] 10.222.0.45 () {54 vars in 965 bytes} [Mon Aug 1 13:59:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:59:31.730048 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37a1e4-a77e-45a3-ae08-f09ea4d39a01 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:31.730512 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37a1e4-a77e-45a3-ae08-f09ea4d39a01 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:31.730997 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37a1e4-a77e-45a3-ae08-f09ea4d39a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:31.747816 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:31.837125 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-4b37a1e4-a77e-45a3-ae08-f09ea4d39a01 None None] MFA Rules not processed for user `6dd44b7a78594288b6aabb393fd6324a`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:59:31.838340 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4b37a1e4-a77e-45a3-ae08-f09ea4d39a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:31.940172 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 661/1315] 10.222.0.45 () {58 vars in 1062 bytes} [Mon Aug 1 13:59:31 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 13:59:31.949553 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc4122-8e99-4bf1-8869-ea3eca1428ed None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:31.951025 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c9fc4122-8e99-4bf1-8869-ea3eca1428ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:32.104013 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc4122-8e99-4bf1-8869-ea3eca1428ed None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:32.216521 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:32.220828 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a55edc1-fbf3-45a3-8f75-3a76e0e1764f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:32.221961 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55edc1-fbf3-45a3-8f75-3a76e0e1764f service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:32.222118 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55edc1-fbf3-45a3-8f75-3a76e0e1764f service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:32.224207 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55edc1-fbf3-45a3-8f75-3a76e0e1764f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:32.224407 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3a55edc1-fbf3-45a3-8f75-3a76e0e1764f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:32.360648 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a55edc1-fbf3-45a3-8f75-3a76e0e1764f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:32.362347 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a55edc1-fbf3-45a3-8f75-3a76e0e1764f service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:32.387395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 655/1316] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 13:59:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 13:59:35.089128 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7fe3e4-b0f8-4b1f-9e48-0c71be3c1692 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:35.089751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7fe3e4-b0f8-4b1f-9e48-0c71be3c1692 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:35.089751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7fe3e4-b0f8-4b1f-9e48-0c71be3c1692 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:35.091090 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 662/1317] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:59:35.096915 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49433e2e-17df-40b5-bebb-8802031a1913 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:35.097653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49433e2e-17df-40b5-bebb-8802031a1913 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:35.097653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49433e2e-17df-40b5-bebb-8802031a1913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:35.188060 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-49433e2e-17df-40b5-bebb-8802031a1913 None None] MFA Rules not processed for user `0cd5be8f747c4089988de317f6307400`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:59:35.189311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-49433e2e-17df-40b5-bebb-8802031a1913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:35.260845 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:35.316226 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 656/1318] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:59:35 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 13:59:35.366267 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7fe3e4-b0f8-4b1f-9e48-0c71be3c1692 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:35.367963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ab7fe3e4-b0f8-4b1f-9e48-0c71be3c1692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:35.500319 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7fe3e4-b0f8-4b1f-9e48-0c71be3c1692 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:35.503436 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:35.595695 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788d7c5-9f0b-45dc-9983-6d43d2d4b00c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:35.596357 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788d7c5-9f0b-45dc-9983-6d43d2d4b00c service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:35.596641 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788d7c5-9f0b-45dc-9983-6d43d2d4b00c service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:35.596911 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788d7c5-9f0b-45dc-9983-6d43d2d4b00c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:35.598587 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6788d7c5-9f0b-45dc-9983-6d43d2d4b00c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:35.737746 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788d7c5-9f0b-45dc-9983-6d43d2d4b00c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:35.739230 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788d7c5-9f0b-45dc-9983-6d43d2d4b00c service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:35.759682 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 663/1319] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 13:59:35 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 13:59:38.690751 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff4a0e-92e0-49c9-9a52-3ac1d434f152 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:38.691349 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff4a0e-92e0-49c9-9a52-3ac1d434f152 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:38.691349 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff4a0e-92e0-49c9-9a52-3ac1d434f152 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:38.692738 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 657/1320] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:59:38.699193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebe676-5138-47eb-9e46-fc8b2d9c6c21 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:38.699779 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebe676-5138-47eb-9e46-fc8b2d9c6c21 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:38.699779 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebe676-5138-47eb-9e46-fc8b2d9c6c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:38.748972 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:38.779451 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-aaebe676-5138-47eb-9e46-fc8b2d9c6c21 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:59:38.780742 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-aaebe676-5138-47eb-9e46-fc8b2d9c6c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:38.907808 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 664/1321] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:59:38 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 13:59:38.947956 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff4a0e-92e0-49c9-9a52-3ac1d434f152 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:38.949622 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-57ff4a0e-92e0-49c9-9a52-3ac1d434f152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:39.066074 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff4a0e-92e0-49c9-9a52-3ac1d434f152 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:39.159857 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f8e13-9735-4a16-b72e-2c73a3b3bf8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:39.160446 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f8e13-9735-4a16-b72e-2c73a3b3bf8a service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:39.160731 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f8e13-9735-4a16-b72e-2c73a3b3bf8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:39.161005 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f8e13-9735-4a16-b72e-2c73a3b3bf8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:39.162751 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1a6f8e13-9735-4a16-b72e-2c73a3b3bf8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:39.171837 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:39.307964 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f8e13-9735-4a16-b72e-2c73a3b3bf8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:39.309468 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f8e13-9735-4a16-b72e-2c73a3b3bf8a service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:39.328548 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 658/1322] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 13:59:38 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 13:59:42.435477 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffa842-cebd-4409-bfb1-69b19031877f None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:42.436046 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffa842-cebd-4409-bfb1-69b19031877f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:42.436325 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffa842-cebd-4409-bfb1-69b19031877f None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:42.437900 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 665/1323] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:59:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:59:42.444606 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211ca29-51b8-47fa-9be5-31f4b6979d56 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:42.445433 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211ca29-51b8-47fa-9be5-31f4b6979d56 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:42.445596 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211ca29-51b8-47fa-9be5-31f4b6979d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:42.534237 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-3211ca29-51b8-47fa-9be5-31f4b6979d56 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:59:42.535690 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3211ca29-51b8-47fa-9be5-31f4b6979d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:42.615350 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:42.674852 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 659/1324] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:59:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 13:59:42.713590 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ffa842-cebd-4409-bfb1-69b19031877f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:42.715165 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-20ffa842-cebd-4409-bfb1-69b19031877f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:42.845646 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ffa842-cebd-4409-bfb1-69b19031877f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:42.858459 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:42.944883 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479c27e-63cb-41a8-a212-d6f37f3e8635 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:42.945698 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479c27e-63cb-41a8-a212-d6f37f3e8635 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:42.946030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479c27e-63cb-41a8-a212-d6f37f3e8635 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:42.946333 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479c27e-63cb-41a8-a212-d6f37f3e8635 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:42.948076 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e479c27e-63cb-41a8-a212-d6f37f3e8635 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:43.089021 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e479c27e-63cb-41a8-a212-d6f37f3e8635 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:43.090480 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e479c27e-63cb-41a8-a212-d6f37f3e8635 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:43.114677 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 666/1325] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 13:59:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 13:59:45.249167 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e235c-de81-4054-b692-0e71030ec30d None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:45.249742 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e235c-de81-4054-b692-0e71030ec30d None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:45.249742 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e235c-de81-4054-b692-0e71030ec30d None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:45.251193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 660/1326] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:59:45.257539 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d695f-b794-44cf-a5d8-5f9089360b28 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:45.258072 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d695f-b794-44cf-a5d8-5f9089360b28 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:45.258199 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d695f-b794-44cf-a5d8-5f9089360b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:45.320583 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:45.342171 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8c7d695f-b794-44cf-a5d8-5f9089360b28 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:59:45.343384 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8c7d695f-b794-44cf-a5d8-5f9089360b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:45.475265 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 667/1327] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:59:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 13:59:45.518522 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e235c-de81-4054-b692-0e71030ec30d None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:45.521199 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-899e235c-de81-4054-b692-0e71030ec30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:45.666914 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e235c-de81-4054-b692-0e71030ec30d None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:45.768338 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c701ddb7-3273-446c-9d2f-6772b5013a5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:45.769134 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701ddb7-3273-446c-9d2f-6772b5013a5f service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:45.769254 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701ddb7-3273-446c-9d2f-6772b5013a5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:45.769518 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701ddb7-3273-446c-9d2f-6772b5013a5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:45.771364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c701ddb7-3273-446c-9d2f-6772b5013a5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:45.792775 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:45.919171 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c701ddb7-3273-446c-9d2f-6772b5013a5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:45.921045 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c701ddb7-3273-446c-9d2f-6772b5013a5f service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:45.943078 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 661/1328] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 13:59:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 13:59:48.064120 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c807734-f3fb-4fb9-ab08-fd15fc2390ad None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:48.064817 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c807734-f3fb-4fb9-ab08-fd15fc2390ad None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:48.064817 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c807734-f3fb-4fb9-ab08-fd15fc2390ad None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:48.066115 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 668/1329] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:59:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:59:48.072233 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb538a-d5f4-46df-aedf-167ea6eed797 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:48.073057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb538a-d5f4-46df-aedf-167ea6eed797 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:48.073057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb538a-d5f4-46df-aedf-167ea6eed797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:48.150685 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-54bb538a-d5f4-46df-aedf-167ea6eed797 None None] MFA Rules not processed for user `0cd5be8f747c4089988de317f6307400`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:59:48.152436 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-54bb538a-d5f4-46df-aedf-167ea6eed797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:48.225545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:48.265007 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 662/1330] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:59:48 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 13:59:48.305704 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c807734-f3fb-4fb9-ab08-fd15fc2390ad None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:48.307268 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8c807734-f3fb-4fb9-ab08-fd15fc2390ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:48.443201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c807734-f3fb-4fb9-ab08-fd15fc2390ad None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:48.468252 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:48.564218 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e183f3-3ff3-40d3-a419-4e1271124417 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:48.564849 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e183f3-3ff3-40d3-a419-4e1271124417 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:48.565133 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e183f3-3ff3-40d3-a419-4e1271124417 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:48.565409 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e183f3-3ff3-40d3-a419-4e1271124417 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:48.567152 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d6e183f3-3ff3-40d3-a419-4e1271124417 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:48.729339 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e183f3-3ff3-40d3-a419-4e1271124417 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:48.730821 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e183f3-3ff3-40d3-a419-4e1271124417 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:48.752771 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 669/1331] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 13:59:48 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 13:59:51.189252 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3700198-65c5-400d-a85b-d537f113773d None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:51.190113 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3700198-65c5-400d-a85b-d537f113773d None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:51.190113 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3700198-65c5-400d-a85b-d537f113773d None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:51.191790 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 663/1332] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:59:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:59:51.199308 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e2288-93b4-40b3-8784-1708a230aa74 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:51.199915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e2288-93b4-40b3-8784-1708a230aa74 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:51.200296 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e2288-93b4-40b3-8784-1708a230aa74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:51.309578 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:51.313869 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-0a2e2288-93b4-40b3-8784-1708a230aa74 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:59:51.316055 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0a2e2288-93b4-40b3-8784-1708a230aa74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:51.444918 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 670/1333] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:59:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 13:59:51.487532 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3700198-65c5-400d-a85b-d537f113773d None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:51.489191 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a3700198-65c5-400d-a85b-d537f113773d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:51.639537 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3700198-65c5-400d-a85b-d537f113773d None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:51.723775 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88599957-bf4b-4e99-bf5d-57a3a421f712 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:51.724449 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88599957-bf4b-4e99-bf5d-57a3a421f712 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:51.724731 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88599957-bf4b-4e99-bf5d-57a3a421f712 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:51.725001 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88599957-bf4b-4e99-bf5d-57a3a421f712 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:51.726719 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-88599957-bf4b-4e99-bf5d-57a3a421f712 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:51.761593 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:51.856507 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88599957-bf4b-4e99-bf5d-57a3a421f712 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:51.858056 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88599957-bf4b-4e99-bf5d-57a3a421f712 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:51.881509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 664/1334] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 13:59:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 13:59:53.968934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0acb664-3741-4cd6-9e99-10bf2840118b None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:53.971024 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0acb664-3741-4cd6-9e99-10bf2840118b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:53.971024 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0acb664-3741-4cd6-9e99-10bf2840118b None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:53.972019 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 671/1335] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:59:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:59:53.978844 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342672a-865f-4816-a8e2-7c7fea8dbb55 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:53.979351 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342672a-865f-4816-a8e2-7c7fea8dbb55 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:53.979445 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342672a-865f-4816-a8e2-7c7fea8dbb55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:54.086387 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-e342672a-865f-4816-a8e2-7c7fea8dbb55 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:59:54.088235 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e342672a-865f-4816-a8e2-7c7fea8dbb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:54.181741 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:54.215728 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 665/1336] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:59:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 13:59:54.258151 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0acb664-3741-4cd6-9e99-10bf2840118b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:54.259736 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d0acb664-3741-4cd6-9e99-10bf2840118b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:54.384561 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0acb664-3741-4cd6-9e99-10bf2840118b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:54.426723 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:54.479827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c174175-74d1-492d-bc0a-1caa1c3519fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:54.480537 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c174175-74d1-492d-bc0a-1caa1c3519fa service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:54.480798 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c174175-74d1-492d-bc0a-1caa1c3519fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:54.481068 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c174175-74d1-492d-bc0a-1caa1c3519fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:54.482763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0c174175-74d1-492d-bc0a-1caa1c3519fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:54.617868 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c174175-74d1-492d-bc0a-1caa1c3519fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:54.619361 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c174175-74d1-492d-bc0a-1caa1c3519fa service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:54.642805 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 672/1337] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 13:59:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 13:59:57.086909 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe950761-c206-4666-84a4-bea8e5e058f1 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:57.087531 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe950761-c206-4666-84a4-bea8e5e058f1 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:57.087531 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe950761-c206-4666-84a4-bea8e5e058f1 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:57.088838 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 666/1338] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 13:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:59:57.094960 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acd4d9-e7fb-48fe-ae36-a6be9ffa165c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:57.095479 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acd4d9-e7fb-48fe-ae36-a6be9ffa165c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:57.095583 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acd4d9-e7fb-48fe-ae36-a6be9ffa165c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:57.170928 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-05acd4d9-e7fb-48fe-ae36-a6be9ffa165c None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:59:57.172147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-05acd4d9-e7fb-48fe-ae36-a6be9ffa165c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:57.180639 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:57.272088 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 673/1339] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 13:59:57 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 13:59:57.309397 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe950761-c206-4666-84a4-bea8e5e058f1 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:57.310971 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-fe950761-c206-4666-84a4-bea8e5e058f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:57.425234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe950761-c206-4666-84a4-bea8e5e058f1 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:57.511412 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd36f33-28ed-4b65-8397-00f3d30d6ee0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:57.511705 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd36f33-28ed-4b65-8397-00f3d30d6ee0 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:57.511940 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd36f33-28ed-4b65-8397-00f3d30d6ee0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:57.512214 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd36f33-28ed-4b65-8397-00f3d30d6ee0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:57.513953 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7bd36f33-28ed-4b65-8397-00f3d30d6ee0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:57.561770 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:57.678879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd36f33-28ed-4b65-8397-00f3d30d6ee0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:57.680305 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd36f33-28ed-4b65-8397-00f3d30d6ee0 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:57.701824 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 667/1340] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 13:59:57 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:00:01.945936 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce682925-7036-4362-b72c-4a0a0ff46904 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:01.946517 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce682925-7036-4362-b72c-4a0a0ff46904 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:01.946517 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce682925-7036-4362-b72c-4a0a0ff46904 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:01.948020 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 674/1341] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 14:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:00:01.953379 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe982fb-b276-42ad-87c0-644496390205 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:01.953893 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe982fb-b276-42ad-87c0-644496390205 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:01.954006 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe982fb-b276-42ad-87c0-644496390205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:02.032254 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-dfe982fb-b276-42ad-87c0-644496390205 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:02.033572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-dfe982fb-b276-42ad-87c0-644496390205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:02.148128 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:02.165072 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 668/1342] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 14:00:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:00:02.204978 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce682925-7036-4362-b72c-4a0a0ff46904 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:00:02.206579 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ce682925-7036-4362-b72c-4a0a0ff46904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:02.330337 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce682925-7036-4362-b72c-4a0a0ff46904 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:00:02.388313 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:02.438074 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb32d8d-ddf4-4822-be78-902493881083 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:00:02.438953 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb32d8d-ddf4-4822-be78-902493881083 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:02.439233 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb32d8d-ddf4-4822-be78-902493881083 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:02.439504 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb32d8d-ddf4-4822-be78-902493881083 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:02.441214 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-beb32d8d-ddf4-4822-be78-902493881083 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:02.592397 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb32d8d-ddf4-4822-be78-902493881083 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:00:02.594062 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb32d8d-ddf4-4822-be78-902493881083 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:00:02.619821 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 675/1343] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:00:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:00:05.729389 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c220c4-3009-48b6-bc09-0b6ad017671a None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:05.730031 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c220c4-3009-48b6-bc09-0b6ad017671a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:05.730031 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c220c4-3009-48b6-bc09-0b6ad017671a None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:05.731271 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 669/1344] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 14:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:00:05.736824 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fdd6ad-78a6-4dec-8f18-9647f8c525f6 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:05.737495 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fdd6ad-78a6-4dec-8f18-9647f8c525f6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:05.737495 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fdd6ad-78a6-4dec-8f18-9647f8c525f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:05.817053 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-48fdd6ad-78a6-4dec-8f18-9647f8c525f6 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:05.818173 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-48fdd6ad-78a6-4dec-8f18-9647f8c525f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:05.844656 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:05.936813 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 676/1345] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 14:00:05 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:00:05.978244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c220c4-3009-48b6-bc09-0b6ad017671a None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:00:05.979829 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e7c220c4-3009-48b6-bc09-0b6ad017671a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:06.104142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c220c4-3009-48b6-bc09-0b6ad017671a None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:00:06.202216 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c352e-0f40-4577-b937-8339648d2c9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:00:06.202861 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c352e-0f40-4577-b937-8339648d2c9c service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:06.203134 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c352e-0f40-4577-b937-8339648d2c9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:06.203398 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c352e-0f40-4577-b937-8339648d2c9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:06.205138 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e44c352e-0f40-4577-b937-8339648d2c9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:06.265801 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:06.353635 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c352e-0f40-4577-b937-8339648d2c9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:00:06.358170 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c352e-0f40-4577-b937-8339648d2c9c service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:00:06.389604 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 670/1346] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:00:05 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:00:11.447644 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc215cd-7a7a-40f8-bbae-77b258288a6d None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:11.448321 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc215cd-7a7a-40f8-bbae-77b258288a6d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:11.448472 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc215cd-7a7a-40f8-bbae-77b258288a6d None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:11.450726 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 677/1347] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 14:00:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:00:11.457387 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d409e-c3e5-49cd-bc6d-68b7db32ed41 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:11.457977 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d409e-c3e5-49cd-bc6d-68b7db32ed41 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:11.457977 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d409e-c3e5-49cd-bc6d-68b7db32ed41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:11.587462 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-7f9d409e-c3e5-49cd-bc6d-68b7db32ed41 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:11.589665 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7f9d409e-c3e5-49cd-bc6d-68b7db32ed41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:11.757459 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:11.761987 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 671/1348] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 14:00:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:00:11.810101 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc215cd-7a7a-40f8-bbae-77b258288a6d None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:00:11.812164 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ccc215cd-7a7a-40f8-bbae-77b258288a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:11.992729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc215cd-7a7a-40f8-bbae-77b258288a6d None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:00:12.075254 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:12.122581 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3131500-68e3-4f9f-adec-018419d9e78b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:00:12.123704 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3131500-68e3-4f9f-adec-018419d9e78b service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:12.124175 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3131500-68e3-4f9f-adec-018419d9e78b service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:12.124739 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3131500-68e3-4f9f-adec-018419d9e78b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:12.127782 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b3131500-68e3-4f9f-adec-018419d9e78b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:12.343887 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3131500-68e3-4f9f-adec-018419d9e78b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:00:12.346742 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3131500-68e3-4f9f-adec-018419d9e78b service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:00:12.386332 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 678/1349] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:00:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:00:18.409971 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5fdc45-0736-4385-9d19-13027411ff06 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:18.410758 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5fdc45-0736-4385-9d19-13027411ff06 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:18.410758 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5fdc45-0736-4385-9d19-13027411ff06 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:18.412188 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 672/1350] 10.222.0.45 () {54 vars in 921 bytes} [Mon Aug 1 14:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:00:18.422505 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a012f1b-ea83-4a00-ac30-4f3458284d74 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:18.423769 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a012f1b-ea83-4a00-ac30-4f3458284d74 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:18.424238 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a012f1b-ea83-4a00-ac30-4f3458284d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:18.566905 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-9a012f1b-ea83-4a00-ac30-4f3458284d74 None None] MFA Rules not processed for user `58380e30789041e4ad9e8652bc628658`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:18.568568 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9a012f1b-ea83-4a00-ac30-4f3458284d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:18.596849 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:18.901463 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 679/1351] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Aug 1 14:00:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 480 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:00:24.041975 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823a54b-b468-463b-b74b-3e17bb40efe3 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:24.043044 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823a54b-b468-463b-b74b-3e17bb40efe3 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:24.043533 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823a54b-b468-463b-b74b-3e17bb40efe3 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:24.045307 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 673/1352] 10.222.0.45 () {54 vars in 921 bytes} [Mon Aug 1 14:00:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:00:24.051155 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bea17-a5a8-47b6-8800-a231887364cc None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:24.051912 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bea17-a5a8-47b6-8800-a231887364cc None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:24.051912 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bea17-a5a8-47b6-8800-a231887364cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:24.162111 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-406bea17-a5a8-47b6-8800-a231887364cc None None] MFA Rules not processed for user `58380e30789041e4ad9e8652bc628658`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:24.163353 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-406bea17-a5a8-47b6-8800-a231887364cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:24.277309 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:24.330501 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 680/1353] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Aug 1 14:00:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:00:25.633604 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30137042-a9a6-40c9-8560-98f17acd82d2 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:25.634230 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30137042-a9a6-40c9-8560-98f17acd82d2 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:25.634230 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30137042-a9a6-40c9-8560-98f17acd82d2 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:25.635754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 674/1354] 10.222.0.45 () {54 vars in 921 bytes} [Mon Aug 1 14:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:00:25.641618 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0849bf2-deb2-4516-b2dd-8a816b6e369f None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:25.642156 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0849bf2-deb2-4516-b2dd-8a816b6e369f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:25.642279 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0849bf2-deb2-4516-b2dd-8a816b6e369f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:25.744862 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c0849bf2-deb2-4516-b2dd-8a816b6e369f None None] MFA Rules not processed for user `58380e30789041e4ad9e8652bc628658`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:25.746066 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c0849bf2-deb2-4516-b2dd-8a816b6e369f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:25.887377 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 681/1355] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Aug 1 14:00:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:00:29.374726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b77c83-d645-4b7c-8327-e3e4dd4b8ac3 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:29.375332 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b77c83-d645-4b7c-8327-e3e4dd4b8ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:29.375332 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b77c83-d645-4b7c-8327-e3e4dd4b8ac3 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:29.376765 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 675/1356] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 14:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:00:29.382689 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce34aa-e456-4c14-947f-eff771b7a384 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:29.383200 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce34aa-e456-4c14-947f-eff771b7a384 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:29.383338 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce34aa-e456-4c14-947f-eff771b7a384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:29.427188 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:29.477930 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c9ce34aa-e456-4c14-947f-eff771b7a384 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:29.479222 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c9ce34aa-e456-4c14-947f-eff771b7a384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:29.686359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 682/1357] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 14:00:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:00:29.743169 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b6461-402f-4aeb-b530-a7bc9be7f824 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:29.743998 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b6461-402f-4aeb-b530-a7bc9be7f824 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:29.744571 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b6461-402f-4aeb-b530-a7bc9be7f824 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:29.748150 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 676/1358] 10.222.0.45 () {54 vars in 962 bytes} [Mon Aug 1 14:00:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:00:29.759529 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486a3d5-5e1b-4d70-ab54-6673204b1196 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:29.760684 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486a3d5-5e1b-4d70-ab54-6673204b1196 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:29.761166 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486a3d5-5e1b-4d70-ab54-6673204b1196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:29.881829 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-3486a3d5-5e1b-4d70-ab54-6673204b1196 None None] MFA Rules not processed for user `5b3adc14dcaf4ac79b43bb97fa1ba1d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:29.883402 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3486a3d5-5e1b-4d70-ab54-6673204b1196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:29.974387 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:30.060559 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 683/1359] 10.222.0.45 () {58 vars in 1059 bytes} [Mon Aug 1 14:00:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:00:30.071687 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b6461-402f-4aeb-b530-a7bc9be7f824 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:00:30.073550 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d95b6461-402f-4aeb-b530-a7bc9be7f824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:30.289380 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b6461-402f-4aeb-b530-a7bc9be7f824 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:00:30.389115 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:30.446168 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a264e7-2160-47fe-ba46-4183fbd7074b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:00:30.446168 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a264e7-2160-47fe-ba46-4183fbd7074b service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:30.446812 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a264e7-2160-47fe-ba46-4183fbd7074b service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:30.446812 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a264e7-2160-47fe-ba46-4183fbd7074b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:30.448306 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d2a264e7-2160-47fe-ba46-4183fbd7074b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:30.567578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a264e7-2160-47fe-ba46-4183fbd7074b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:00:30.569234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a264e7-2160-47fe-ba46-4183fbd7074b service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:00:30.590727 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 677/1360] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:00:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:00:33.136881 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec925de-fc98-4d31-b5be-d82d2d8787d3 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:33.137686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec925de-fc98-4d31-b5be-d82d2d8787d3 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:33.137686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec925de-fc98-4d31-b5be-d82d2d8787d3 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:33.139018 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 684/1361] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 14:00:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:00:33.145862 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f0049-02a4-42e7-b719-397e4569a92a None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:33.147190 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f0049-02a4-42e7-b719-397e4569a92a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:33.148006 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f0049-02a4-42e7-b719-397e4569a92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:33.262642 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-905f0049-02a4-42e7-b719-397e4569a92a None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:33.263879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-905f0049-02a4-42e7-b719-397e4569a92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:33.314134 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:33.407579 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 678/1362] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 14:00:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:00:33.456464 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec925de-fc98-4d31-b5be-d82d2d8787d3 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:00:33.457976 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0ec925de-fc98-4d31-b5be-d82d2d8787d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:33.691052 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec925de-fc98-4d31-b5be-d82d2d8787d3 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:00:33.702993 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:33.851213 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de1bb40-435b-43be-be61-1ab0075b8389 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:00:33.852218 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de1bb40-435b-43be-be61-1ab0075b8389 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:33.852694 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de1bb40-435b-43be-be61-1ab0075b8389 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:33.852980 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de1bb40-435b-43be-be61-1ab0075b8389 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:33.854891 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3de1bb40-435b-43be-be61-1ab0075b8389 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:33.978298 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de1bb40-435b-43be-be61-1ab0075b8389 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:00:33.979948 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de1bb40-435b-43be-be61-1ab0075b8389 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:00:34.004419 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 685/1363] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:00:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:00:36.735286 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41a407-1b12-4074-b1a2-7f840e288c56 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:36.736196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41a407-1b12-4074-b1a2-7f840e288c56 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:36.736766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41a407-1b12-4074-b1a2-7f840e288c56 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:36.739447 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 679/1364] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 14:00:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:00:36.749344 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9dec04-90e9-4c78-95f5-b560f236c358 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:36.749344 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9dec04-90e9-4c78-95f5-b560f236c358 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:36.749344 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9dec04-90e9-4c78-95f5-b560f236c358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:36.778597 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:36.878684 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8c9dec04-90e9-4c78-95f5-b560f236c358 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:36.878947 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8c9dec04-90e9-4c78-95f5-b560f236c358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:37.011573 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 686/1365] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 14:00:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:00:37.059045 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac7837-0466-4d60-aec2-1aa4b0c8e9fa None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:37.059632 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac7837-0466-4d60-aec2-1aa4b0c8e9fa None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:37.059902 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac7837-0466-4d60-aec2-1aa4b0c8e9fa None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:37.061499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 680/1366] 10.222.0.45 () {54 vars in 962 bytes} [Mon Aug 1 14:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:00:37.067575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351cc7c6-7cbf-4182-986f-f45e075406a9 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:37.067996 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351cc7c6-7cbf-4182-986f-f45e075406a9 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:37.068419 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351cc7c6-7cbf-4182-986f-f45e075406a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:37.224827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-351cc7c6-7cbf-4182-986f-f45e075406a9 None None] MFA Rules not processed for user `5b3adc14dcaf4ac79b43bb97fa1ba1d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:37.226852 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-351cc7c6-7cbf-4182-986f-f45e075406a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:37.301902 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:37.436576 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 687/1367] 10.222.0.45 () {58 vars in 1059 bytes} [Mon Aug 1 14:00:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:00:37.444900 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac7837-0466-4d60-aec2-1aa4b0c8e9fa None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:00:37.446834 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-99ac7837-0466-4d60-aec2-1aa4b0c8e9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:37.640025 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac7837-0466-4d60-aec2-1aa4b0c8e9fa None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:00:37.681507 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:37.825364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbcfe1-89cb-425d-b42f-16448f0f1667 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:00:37.826043 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbcfe1-89cb-425d-b42f-16448f0f1667 service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:37.826043 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbcfe1-89cb-425d-b42f-16448f0f1667 service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:37.826365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbcfe1-89cb-425d-b42f-16448f0f1667 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:37.829262 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b9fbcfe1-89cb-425d-b42f-16448f0f1667 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:37.965930 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fbcfe1-89cb-425d-b42f-16448f0f1667 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:00:37.968364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fbcfe1-89cb-425d-b42f-16448f0f1667 service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:00:38.003299 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 681/1368] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:00:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:00:38.789796 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8880bb6-6316-4c5d-8553-5b9c4ae4867f None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:38.790715 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8880bb6-6316-4c5d-8553-5b9c4ae4867f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:38.790715 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8880bb6-6316-4c5d-8553-5b9c4ae4867f None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:38.791827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 688/1369] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 14:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:00:38.797492 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6775b19-fa9b-4506-b4fa-972415c13436 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:38.797782 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6775b19-fa9b-4506-b4fa-972415c13436 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:38.797954 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6775b19-fa9b-4506-b4fa-972415c13436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:38.837331 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:38.888494 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-e6775b19-fa9b-4506-b4fa-972415c13436 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:38.889933 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e6775b19-fa9b-4506-b4fa-972415c13436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:39.033627 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 682/1370] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 14:00:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:00:39.407853 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b42b2-2657-414f-ac40-426e88b97c4a None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:39.408446 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b42b2-2657-414f-ac40-426e88b97c4a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:39.408888 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b42b2-2657-414f-ac40-426e88b97c4a None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:39.410575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 689/1371] 10.222.0.45 () {54 vars in 964 bytes} [Mon Aug 1 14:00:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:00:39.416111 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cdaf9-0adf-427c-9b50-332df9249e51 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:39.416111 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cdaf9-0adf-427c-9b50-332df9249e51 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:39.417134 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cdaf9-0adf-427c-9b50-332df9249e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:39.529443 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-1e7cdaf9-0adf-427c-9b50-332df9249e51 None None] MFA Rules not processed for user `8f4c3d46ac7243aeb722a4410f40d846`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:39.531086 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1e7cdaf9-0adf-427c-9b50-332df9249e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:39.603918 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:39.658859 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 683/1372] 10.222.0.45 () {58 vars in 1061 bytes} [Mon Aug 1 14:00:39 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:00:39.666554 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b42b2-2657-414f-ac40-426e88b97c4a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:00:39.668106 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0a8b42b2-2657-414f-ac40-426e88b97c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:39.843403 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b42b2-2657-414f-ac40-426e88b97c4a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:00:39.938001 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:40.005832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35791fb-6ee9-486c-9467-72b2824be462 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:00:40.005832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35791fb-6ee9-486c-9467-72b2824be462 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:40.005832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35791fb-6ee9-486c-9467-72b2824be462 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:40.005832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35791fb-6ee9-486c-9467-72b2824be462 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:40.006710 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e35791fb-6ee9-486c-9467-72b2824be462 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:40.196506 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35791fb-6ee9-486c-9467-72b2824be462 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:00:40.200495 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35791fb-6ee9-486c-9467-72b2824be462 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:00:40.251872 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 690/1373] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:00:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:00:40.485792 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe689b-f2a0-4b5a-8f28-084123eaf4f4 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:40.486595 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe689b-f2a0-4b5a-8f28-084123eaf4f4 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:40.486595 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe689b-f2a0-4b5a-8f28-084123eaf4f4 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:40.488024 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 684/1374] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 14:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:00:40.494470 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d9cac-6054-46a2-adff-514e80e38521 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:40.494908 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d9cac-6054-46a2-adff-514e80e38521 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:40.495369 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d9cac-6054-46a2-adff-514e80e38521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:40.594294 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-bd6d9cac-6054-46a2-adff-514e80e38521 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:40.595628 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-bd6d9cac-6054-46a2-adff-514e80e38521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:40.621258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:40.734535 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 691/1375] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 14:00:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:00:40.794897 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe689b-f2a0-4b5a-8f28-084123eaf4f4 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:00:40.796660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-56fe689b-f2a0-4b5a-8f28-084123eaf4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:41.014918 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe689b-f2a0-4b5a-8f28-084123eaf4f4 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:00:41.123853 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:41.212748 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ec6c0-34a6-4646-b0d9-06f3d0b13d3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:00:41.213390 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ec6c0-34a6-4646-b0d9-06f3d0b13d3f service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:41.213714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ec6c0-34a6-4646-b0d9-06f3d0b13d3f service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:41.213982 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ec6c0-34a6-4646-b0d9-06f3d0b13d3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:41.215690 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-184ec6c0-34a6-4646-b0d9-06f3d0b13d3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:41.359572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ec6c0-34a6-4646-b0d9-06f3d0b13d3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:00:41.361945 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ec6c0-34a6-4646-b0d9-06f3d0b13d3f service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:00:41.393002 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 685/1376] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:00:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:00:43.634804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7528df-b3fb-42a9-97b8-08633195d570 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:43.635482 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7528df-b3fb-42a9-97b8-08633195d570 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:43.635482 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7528df-b3fb-42a9-97b8-08633195d570 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:43.642022 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 692/1377] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 14:00:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:00:43.643123 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b97798-e738-4066-86f3-060bac054f17 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:43.643581 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b97798-e738-4066-86f3-060bac054f17 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:43.643676 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b97798-e738-4066-86f3-060bac054f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:43.721636 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-38b97798-e738-4066-86f3-060bac054f17 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:43.723519 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-38b97798-e738-4066-86f3-060bac054f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:43.735210 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:43.835401 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 686/1378] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 14:00:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:00:43.880001 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7528df-b3fb-42a9-97b8-08633195d570 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:00:43.881697 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fb7528df-b3fb-42a9-97b8-08633195d570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:44.109961 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7528df-b3fb-42a9-97b8-08633195d570 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:00:44.124319 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:44.288625 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d0f138-c7db-4dff-8876-dfe2680016b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:00:44.289612 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0f138-c7db-4dff-8876-dfe2680016b4 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:44.290097 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0f138-c7db-4dff-8876-dfe2680016b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:44.290651 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0f138-c7db-4dff-8876-dfe2680016b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:44.292712 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c7d0f138-c7db-4dff-8876-dfe2680016b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:44.421881 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d0f138-c7db-4dff-8876-dfe2680016b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:00:44.424566 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d0f138-c7db-4dff-8876-dfe2680016b4 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:00:44.457021 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 693/1379] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:00:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:00:46.877969 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61daa68-3859-46e4-b624-070195e96531 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:46.878495 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61daa68-3859-46e4-b624-070195e96531 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:46.878495 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61daa68-3859-46e4-b624-070195e96531 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:46.880049 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 687/1380] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 14:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:00:46.885765 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0556a6-90a0-418a-ad22-c88027166abf None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:46.886330 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0556a6-90a0-418a-ad22-c88027166abf None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:46.886330 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0556a6-90a0-418a-ad22-c88027166abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:46.898402 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:46.979848 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-4a0556a6-90a0-418a-ad22-c88027166abf None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:46.981154 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4a0556a6-90a0-418a-ad22-c88027166abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:47.136444 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 694/1381] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 14:00:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:00:47.177931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61daa68-3859-46e4-b624-070195e96531 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:00:47.179895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-f61daa68-3859-46e4-b624-070195e96531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:47.378515 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:47.392191 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61daa68-3859-46e4-b624-070195e96531 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:00:47.537626 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364ea60-5dcd-4ed0-9b6c-f1dbd57bc656 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:00:47.538491 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364ea60-5dcd-4ed0-9b6c-f1dbd57bc656 service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:47.538768 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364ea60-5dcd-4ed0-9b6c-f1dbd57bc656 service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:47.539320 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364ea60-5dcd-4ed0-9b6c-f1dbd57bc656 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:47.543542 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a364ea60-5dcd-4ed0-9b6c-f1dbd57bc656 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:47.675010 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a364ea60-5dcd-4ed0-9b6c-f1dbd57bc656 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:00:47.677202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a364ea60-5dcd-4ed0-9b6c-f1dbd57bc656 service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:00:47.695481 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:47.707913 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 688/1382] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:00:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:00:50.077730 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b855e-2823-41a4-bb68-2673ef04a8de None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:50.078797 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b855e-2823-41a4-bb68-2673ef04a8de None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:50.078797 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b855e-2823-41a4-bb68-2673ef04a8de None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:50.080891 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 695/1383] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 14:00:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:00:50.088716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b68bf3-5fd1-41eb-9e82-cfc10efc9630 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:50.088716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b68bf3-5fd1-41eb-9e82-cfc10efc9630 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:50.088716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b68bf3-5fd1-41eb-9e82-cfc10efc9630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:50.200008 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-93b68bf3-5fd1-41eb-9e82-cfc10efc9630 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:50.202173 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-93b68bf3-5fd1-41eb-9e82-cfc10efc9630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:50.355118 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 689/1384] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 14:00:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:00:50.409761 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b855e-2823-41a4-bb68-2673ef04a8de None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:00:50.412033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-af4b855e-2823-41a4-bb68-2673ef04a8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:50.556704 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:50.646443 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b855e-2823-41a4-bb68-2673ef04a8de None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:00:50.814357 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd3ffa-f8c8-49b4-a39b-680fb959b89d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:00:50.815043 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd3ffa-f8c8-49b4-a39b-680fb959b89d service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:50.815322 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd3ffa-f8c8-49b4-a39b-680fb959b89d service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:50.815596 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd3ffa-f8c8-49b4-a39b-680fb959b89d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:50.817482 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f5bd3ffa-f8c8-49b4-a39b-680fb959b89d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:50.898122 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:50.953695 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bd3ffa-f8c8-49b4-a39b-680fb959b89d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:00:50.955632 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bd3ffa-f8c8-49b4-a39b-680fb959b89d service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:00:50.980025 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 696/1385] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:00:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:00:53.832640 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54714c-a6b6-44df-ad73-edd0f275e260 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:53.833276 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54714c-a6b6-44df-ad73-edd0f275e260 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:53.833276 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54714c-a6b6-44df-ad73-edd0f275e260 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:53.835107 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 690/1386] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 14:00:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:00:53.843004 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c50742-5c31-4f49-a864-0f9644252df5 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:53.843666 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c50742-5c31-4f49-a864-0f9644252df5 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:53.843666 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c50742-5c31-4f49-a864-0f9644252df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:53.943725 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-26c50742-5c31-4f49-a864-0f9644252df5 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:53.945275 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-26c50742-5c31-4f49-a864-0f9644252df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:54.145732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 697/1387] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 14:00:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:00:54.214199 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54714c-a6b6-44df-ad73-edd0f275e260 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:00:54.215993 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-2b54714c-a6b6-44df-ad73-edd0f275e260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:54.366308 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:54.500420 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54714c-a6b6-44df-ad73-edd0f275e260 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:00:54.715713 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44378d-b8a3-438f-a2b1-7ea86daaf91e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:00:54.716488 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44378d-b8a3-438f-a2b1-7ea86daaf91e service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:54.716880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44378d-b8a3-438f-a2b1-7ea86daaf91e service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:54.717042 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44378d-b8a3-438f-a2b1-7ea86daaf91e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:54.719732 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1e44378d-b8a3-438f-a2b1-7ea86daaf91e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:54.770848 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:54.835628 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44378d-b8a3-438f-a2b1-7ea86daaf91e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:00:54.837306 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44378d-b8a3-438f-a2b1-7ea86daaf91e service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:00:54.862794 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 691/1388] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:00:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 649 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:00:56.717795 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c49f0-0c4d-47e0-9700-6a4856ad7398 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:56.718461 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c49f0-0c4d-47e0-9700-6a4856ad7398 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:56.718461 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c49f0-0c4d-47e0-9700-6a4856ad7398 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:56.719729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 698/1389] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 14:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:00:56.725260 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062f99a-ea88-4e73-9739-4f6ac53555e5 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:56.725743 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062f99a-ea88-4e73-9739-4f6ac53555e5 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:56.725944 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062f99a-ea88-4e73-9739-4f6ac53555e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:56.796036 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-f062f99a-ea88-4e73-9739-4f6ac53555e5 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:56.797245 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-f062f99a-ea88-4e73-9739-4f6ac53555e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:56.914799 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 692/1390] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 14:00:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:00:56.974201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ff087-083c-4ab7-856d-30547fedb213 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:56.974476 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ff087-083c-4ab7-856d-30547fedb213 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:56.974667 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ff087-083c-4ab7-856d-30547fedb213 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:56.976307 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 699/1391] 10.222.0.45 () {54 vars in 966 bytes} [Mon Aug 1 14:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:00:56.982820 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a93e6-c3a8-4a3b-ad6b-602f66886289 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:56.983221 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a93e6-c3a8-4a3b-ad6b-602f66886289 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:56.983596 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a93e6-c3a8-4a3b-ad6b-602f66886289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:57.001717 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:57.061785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665bee4-6222-4000-b04f-bb201065abef None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:57.061987 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665bee4-6222-4000-b04f-bb201065abef None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:57.062271 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665bee4-6222-4000-b04f-bb201065abef None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:57.063670 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 700/1392] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 14:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:00:57.069350 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126da24-9b87-43cd-a0c7-646437de5536 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:57.069709 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126da24-9b87-43cd-a0c7-646437de5536 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:57.070069 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126da24-9b87-43cd-a0c7-646437de5536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:57.072162 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-128a93e6-c3a8-4a3b-ad6b-602f66886289 None None] MFA Rules not processed for user `e7317c8829cb4fa1b30711f1db575cab`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:57.073496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-128a93e6-c3a8-4a3b-ad6b-602f66886289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:57.163710 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 693/1393] 10.222.0.45 () {58 vars in 1063 bytes} [Mon Aug 1 14:00:56 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:00:57.171960 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a93e6-c3a8-4a3b-ad6b-602f66886289 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:00:57.180698 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a93e6-c3a8-4a3b-ad6b-602f66886289 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:00:57.238179 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:57.271179 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32747002-28ca-44d3-949d-967abf65d178 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7317c8829cb4fa1b30711f1db575cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:00:57.272015 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32747002-28ca-44d3-949d-967abf65d178 service glance] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:57.272482 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32747002-28ca-44d3-949d-967abf65d178 service glance] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:57.272837 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32747002-28ca-44d3-949d-967abf65d178 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:57.274996 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-32747002-28ca-44d3-949d-967abf65d178 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:57.278063 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-7126da24-9b87-43cd-a0c7-646437de5536 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:57.279258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7126da24-9b87-43cd-a0c7-646437de5536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:57.315374 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:57.408122 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 701/1394] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 14:00:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:00:57.417444 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32747002-28ca-44d3-949d-967abf65d178 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:00:57.418964 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32747002-28ca-44d3-949d-967abf65d178 service glance] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:00:57.441586 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 694/1395] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:00:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:00:57.454115 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7126da24-9b87-43cd-a0c7-646437de5536 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:00:57.455541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7126da24-9b87-43cd-a0c7-646437de5536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:57.606021 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:57.615503 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7126da24-9b87-43cd-a0c7-646437de5536 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:00:57.757826 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e9c75d-ab53-4bf3-8168-0998a144ffcf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:00:57.758525 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9c75d-ab53-4bf3-8168-0998a144ffcf service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:57.758725 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9c75d-ab53-4bf3-8168-0998a144ffcf service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:57.758995 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9c75d-ab53-4bf3-8168-0998a144ffcf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:57.760759 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-08e9c75d-ab53-4bf3-8168-0998a144ffcf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:57.862196 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e9c75d-ab53-4bf3-8168-0998a144ffcf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:00:57.863801 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e9c75d-ab53-4bf3-8168-0998a144ffcf service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:00:57.880520 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:57.884563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 702/1396] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:00:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:00:58.256653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885960a-e5fb-44e8-829f-c8220441c39b None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:58.257552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885960a-e5fb-44e8-829f-c8220441c39b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:58.257911 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885960a-e5fb-44e8-829f-c8220441c39b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:58.341223 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-b885960a-e5fb-44e8-829f-c8220441c39b None None] MFA Rules not processed for user `e07e83c0bdba47d39c09308eba1bdaae`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:58.342478 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b885960a-e5fb-44e8-829f-c8220441c39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:58.445533 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:58.454325 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 695/1397] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Aug 1 14:00:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:00:58.502760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db6f6a-25ad-4b57-8a45-d0fafc677f7d None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:58.503217 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db6f6a-25ad-4b57-8a45-d0fafc677f7d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:58.503594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db6f6a-25ad-4b57-8a45-d0fafc677f7d None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:58.505564 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 703/1398] 10.222.0.45 () {54 vars in 943 bytes} [Mon Aug 1 14:00:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:00:58.512969 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1514f-ce2e-4b6c-b0cc-8ae73107a93a None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:58.513428 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1514f-ce2e-4b6c-b0cc-8ae73107a93a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:58.513781 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1514f-ce2e-4b6c-b0cc-8ae73107a93a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:58.604862 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-ade1514f-ce2e-4b6c-b0cc-8ae73107a93a None None] MFA Rules not processed for user `0cda7f365ba0405daaf86870960f12d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:58.606222 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ade1514f-ce2e-4b6c-b0cc-8ae73107a93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:58.726301 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 696/1399] 10.222.0.45 () {58 vars in 1040 bytes} [Mon Aug 1 14:00:58 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:00:58.736253 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db6f6a-25ad-4b57-8a45-d0fafc677f7d None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:00:58.737835 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-33db6f6a-25ad-4b57-8a45-d0fafc677f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:58.912738 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db6f6a-25ad-4b57-8a45-d0fafc677f7d None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:00:59.010206 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:59.047701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2130dc-df5e-404b-bec0-a56ddfa13627 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cda7f365ba0405daaf86870960f12d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:00:59.048819 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2130dc-df5e-404b-bec0-a56ddfa13627 service swift] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:59.049429 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2130dc-df5e-404b-bec0-a56ddfa13627 service swift] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:59.049952 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2130dc-df5e-404b-bec0-a56ddfa13627 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:59.052977 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8a2130dc-df5e-404b-bec0-a56ddfa13627 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:59.174656 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2130dc-df5e-404b-bec0-a56ddfa13627 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:00:59.176520 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2130dc-df5e-404b-bec0-a56ddfa13627 service swift] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:00:59.186729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 704/1400] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 14:00:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:01:00.082195 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc53022-75de-412a-8cc4-7aeb2b4e5f8e None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:00.083054 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc53022-75de-412a-8cc4-7aeb2b4e5f8e None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:00.083054 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc53022-75de-412a-8cc4-7aeb2b4e5f8e None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:00.084334 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 697/1401] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 14:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:00.091417 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15313217-b884-4938-a25c-90e49c24df78 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:00.092070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15313217-b884-4938-a25c-90e49c24df78 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:00.092311 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15313217-b884-4938-a25c-90e49c24df78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:00.205229 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-15313217-b884-4938-a25c-90e49c24df78 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:00.206510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-15313217-b884-4938-a25c-90e49c24df78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:00.295087 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:00.340248 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 705/1402] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 14:01:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:00.385367 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc53022-75de-412a-8cc4-7aeb2b4e5f8e None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:00.387025 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-bcc53022-75de-412a-8cc4-7aeb2b4e5f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:00.484090 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:00.653673 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc53022-75de-412a-8cc4-7aeb2b4e5f8e None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:00.829160 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0878307c-325e-464d-8632-206b817aa3cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:00.829807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878307c-325e-464d-8632-206b817aa3cc service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:00.830094 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878307c-325e-464d-8632-206b817aa3cc service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:00.830363 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878307c-325e-464d-8632-206b817aa3cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:00.832100 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0878307c-325e-464d-8632-206b817aa3cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:00.856483 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:00.954551 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0878307c-325e-464d-8632-206b817aa3cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:00.958235 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0878307c-325e-464d-8632-206b817aa3cc service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:00.986892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 698/1403] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 602 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:01:03.228378 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4dfb0-c976-44bb-ae3c-344dc0609181 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:03.228983 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4dfb0-c976-44bb-ae3c-344dc0609181 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:03.229392 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4dfb0-c976-44bb-ae3c-344dc0609181 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:03.232559 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 706/1404] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 14:01:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:03.239169 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7cc592-a9be-430f-8a6b-b22eb2586345 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:03.240047 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7cc592-a9be-430f-8a6b-b22eb2586345 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:03.240047 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7cc592-a9be-430f-8a6b-b22eb2586345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:03.337003 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-5e7cc592-a9be-430f-8a6b-b22eb2586345 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:03.338418 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5e7cc592-a9be-430f-8a6b-b22eb2586345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:03.452784 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:03.466205 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 699/1405] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 14:01:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:03.517729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4dfb0-c976-44bb-ae3c-344dc0609181 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:03.519815 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-00f4dfb0-c976-44bb-ae3c-344dc0609181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:03.766106 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4dfb0-c976-44bb-ae3c-344dc0609181 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:03.851886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:03.951406 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d4fe4-ea73-4a69-8cff-fdaf2ed72639 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:03.951873 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d4fe4-ea73-4a69-8cff-fdaf2ed72639 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:03.952154 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d4fe4-ea73-4a69-8cff-fdaf2ed72639 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:03.952446 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d4fe4-ea73-4a69-8cff-fdaf2ed72639 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:03.954191 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0c1d4fe4-ea73-4a69-8cff-fdaf2ed72639 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:04.099317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1d4fe4-ea73-4a69-8cff-fdaf2ed72639 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:04.100952 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1d4fe4-ea73-4a69-8cff-fdaf2ed72639 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:04.125814 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 707/1406] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:01:06.385523 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7496ccd-7449-416d-a730-895b8d7e70a7 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:06.386461 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7496ccd-7449-416d-a730-895b8d7e70a7 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:06.386618 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7496ccd-7449-416d-a730-895b8d7e70a7 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:06.390435 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 700/1407] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 14:01:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:06.396554 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a17c79-26fe-45a4-8c1f-5ade2bfe8350 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:06.397349 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a17c79-26fe-45a4-8c1f-5ade2bfe8350 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:06.397520 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a17c79-26fe-45a4-8c1f-5ade2bfe8350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:06.509816 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b8a17c79-26fe-45a4-8c1f-5ade2bfe8350 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:06.511391 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b8a17c79-26fe-45a4-8c1f-5ade2bfe8350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:06.535409 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:06.663460 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 708/1408] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 14:01:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:01:06.732317 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7496ccd-7449-416d-a730-895b8d7e70a7 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:06.733869 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d7496ccd-7449-416d-a730-895b8d7e70a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:06.957183 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7496ccd-7449-416d-a730-895b8d7e70a7 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:07.082084 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:07.138412 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c82a67-ae4e-4248-8488-fb205e187e1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:07.139324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c82a67-ae4e-4248-8488-fb205e187e1a service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:07.139606 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c82a67-ae4e-4248-8488-fb205e187e1a service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:07.139879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c82a67-ae4e-4248-8488-fb205e187e1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:07.141779 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-80c82a67-ae4e-4248-8488-fb205e187e1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:07.270216 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c82a67-ae4e-4248-8488-fb205e187e1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:07.271966 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c82a67-ae4e-4248-8488-fb205e187e1a service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:07.292658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 701/1409] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:01:16.660795 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=37 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:17.290237 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=65 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:22.399065 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca930cf6-115f-4437-8fa1-fd78b63298a6 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:22.399957 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca930cf6-115f-4437-8fa1-fd78b63298a6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:22.399957 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca930cf6-115f-4437-8fa1-fd78b63298a6 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:22.401018 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 709/1410] 10.222.0.45 () {54 vars in 918 bytes} [Mon Aug 1 14:01:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:22.407288 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c01d1e-3da1-490e-b818-b9aaa3591bb2 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:22.407930 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c01d1e-3da1-490e-b818-b9aaa3591bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:22.407930 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c01d1e-3da1-490e-b818-b9aaa3591bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:22.491652 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-18c01d1e-3da1-490e-b818-b9aaa3591bb2 None None] MFA Rules not processed for user `58380e30789041e4ad9e8652bc628658`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:22.493022 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-18c01d1e-3da1-490e-b818-b9aaa3591bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:22.628706 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 702/1411] 10.222.0.45 () {58 vars in 1015 bytes} [Mon Aug 1 14:01:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:22.658343 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fabad1-1dc4-4480-98a3-a96b99d94972 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:22.658566 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fabad1-1dc4-4480-98a3-a96b99d94972 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:22.658812 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fabad1-1dc4-4480-98a3-a96b99d94972 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:22.660245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 710/1412] 10.222.0.45 () {54 vars in 918 bytes} [Mon Aug 1 14:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:22.665869 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104c5cf-8567-4244-a194-ea87b9a6ccc0 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:22.666164 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104c5cf-8567-4244-a194-ea87b9a6ccc0 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:22.666440 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104c5cf-8567-4244-a194-ea87b9a6ccc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:22.759073 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:22.765370 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-e104c5cf-8567-4244-a194-ea87b9a6ccc0 None None] MFA Rules not processed for user `5b3adc14dcaf4ac79b43bb97fa1ba1d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:22.767504 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e104c5cf-8567-4244-a194-ea87b9a6ccc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:22.941924 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 703/1413] 10.222.0.45 () {58 vars in 1015 bytes} [Mon Aug 1 14:01:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:25.759528 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0316127c-fe04-46fe-9742-87b79c72b8e2 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:25.760499 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0316127c-fe04-46fe-9742-87b79c72b8e2 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:25.760499 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0316127c-fe04-46fe-9742-87b79c72b8e2 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:25.761571 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:25.767266 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b41a8c-fc7f-4b7a-abe7-377833398236 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:25.767748 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b41a8c-fc7f-4b7a-abe7-377833398236 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:25.767913 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b41a8c-fc7f-4b7a-abe7-377833398236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:25.842551 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-98b41a8c-fc7f-4b7a-abe7-377833398236 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:25.843682 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-98b41a8c-fc7f-4b7a-abe7-377833398236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:25.908904 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:25.967403 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:01:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:26.018825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0316127c-fe04-46fe-9742-87b79c72b8e2 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:26.020499 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0316127c-fe04-46fe-9742-87b79c72b8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:26.199773 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0316127c-fe04-46fe-9742-87b79c72b8e2 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:26.290004 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:26.360984 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a136886b-4aa0-4a13-8b55-0ad8394cb45c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:26.361597 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136886b-4aa0-4a13-8b55-0ad8394cb45c service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:26.361870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136886b-4aa0-4a13-8b55-0ad8394cb45c service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:26.362136 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136886b-4aa0-4a13-8b55-0ad8394cb45c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:26.363826 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a136886b-4aa0-4a13-8b55-0ad8394cb45c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:26.478727 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a136886b-4aa0-4a13-8b55-0ad8394cb45c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:26.480632 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a136886b-4aa0-4a13-8b55-0ad8394cb45c service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:26.500899 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 712/1416] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:01:27.480164 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af8434-ff3f-4f7c-8f34-2399400f38d0 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:27.480806 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af8434-ff3f-4f7c-8f34-2399400f38d0 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:27.481193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af8434-ff3f-4f7c-8f34-2399400f38d0 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:27.483775 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:01:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:27.489821 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442efb1a-6804-405f-8700-244164df08ba None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:27.490559 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442efb1a-6804-405f-8700-244164df08ba None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:27.490559 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442efb1a-6804-405f-8700-244164df08ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:27.576169 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-442efb1a-6804-405f-8700-244164df08ba None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:27.577456 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-442efb1a-6804-405f-8700-244164df08ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:27.610195 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:27.698240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:01:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:01:27.854132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af8434-ff3f-4f7c-8f34-2399400f38d0 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:27.855769 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b7af8434-ff3f-4f7c-8f34-2399400f38d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:28.048778 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af8434-ff3f-4f7c-8f34-2399400f38d0 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:28.095175 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:28.175031 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46213db9-7413-4482-a544-2cceaec8b727 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:28.175722 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46213db9-7413-4482-a544-2cceaec8b727 service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:28.175995 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46213db9-7413-4482-a544-2cceaec8b727 service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:28.176261 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46213db9-7413-4482-a544-2cceaec8b727 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:28.178165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-46213db9-7413-4482-a544-2cceaec8b727 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:28.283285 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46213db9-7413-4482-a544-2cceaec8b727 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:28.284962 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46213db9-7413-4482-a544-2cceaec8b727 service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:28.304821 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 706/1419] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:01:29.221615 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fed781-46de-40f2-925f-3e8698e949e1 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:29.222509 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fed781-46de-40f2-925f-3e8698e949e1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:29.222509 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fed781-46de-40f2-925f-3e8698e949e1 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:29.224217 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:01:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:29.230196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf717937-9048-4f4a-a743-b0244c60eb9a None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:29.230711 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf717937-9048-4f4a-a743-b0244c60eb9a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:29.230711 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf717937-9048-4f4a-a743-b0244c60eb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:29.321496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:29.348840 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-bf717937-9048-4f4a-a743-b0244c60eb9a None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:29.350149 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-bf717937-9048-4f4a-a743-b0244c60eb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:29.461543 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:01:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:29.617400 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fed781-46de-40f2-925f-3e8698e949e1 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:29.618941 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d2fed781-46de-40f2-925f-3e8698e949e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:29.776751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fed781-46de-40f2-925f-3e8698e949e1 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:29.789367 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:29.953851 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc8163-eded-47fe-89c5-6e06f1888dba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:29.954539 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc8163-eded-47fe-89c5-6e06f1888dba service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:29.954826 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc8163-eded-47fe-89c5-6e06f1888dba service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:29.955112 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc8163-eded-47fe-89c5-6e06f1888dba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:29.956918 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e5fc8163-eded-47fe-89c5-6e06f1888dba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:30.054335 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fc8163-eded-47fe-89c5-6e06f1888dba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:30.055956 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fc8163-eded-47fe-89c5-6e06f1888dba service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:30.076808 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 715/1422] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:01:30.995336 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf0603-78a6-4e30-83f0-6dfbc7ae6d31 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:30.996113 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf0603-78a6-4e30-83f0-6dfbc7ae6d31 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:30.996113 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf0603-78a6-4e30-83f0-6dfbc7ae6d31 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:30.997349 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:01:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:31.003353 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8ba8c-a879-4018-ab5a-74b0e959b6bf None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:31.004004 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8ba8c-a879-4018-ab5a-74b0e959b6bf None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:31.004004 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8ba8c-a879-4018-ab5a-74b0e959b6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:31.027636 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:31.090779 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-3dd8ba8c-a879-4018-ab5a-74b0e959b6bf None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:31.092490 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3dd8ba8c-a879-4018-ab5a-74b0e959b6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:31.202869 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:01:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:31.354072 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf0603-78a6-4e30-83f0-6dfbc7ae6d31 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:31.355619 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-aebf0603-78a6-4e30-83f0-6dfbc7ae6d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:31.477064 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:31.516439 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf0603-78a6-4e30-83f0-6dfbc7ae6d31 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:31.641447 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2274c7-29e9-4dac-846d-b1243545e09a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:31.642095 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2274c7-29e9-4dac-846d-b1243545e09a service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:31.642350 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2274c7-29e9-4dac-846d-b1243545e09a service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:31.642619 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2274c7-29e9-4dac-846d-b1243545e09a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:31.644298 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-af2274c7-29e9-4dac-846d-b1243545e09a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:31.727783 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:31.744704 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2274c7-29e9-4dac-846d-b1243545e09a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:31.747897 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2274c7-29e9-4dac-846d-b1243545e09a service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:31.768052 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 709/1425] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:01:32.693983 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325111a-261f-4985-91cb-ccc8fa84e7f6 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:32.694782 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325111a-261f-4985-91cb-ccc8fa84e7f6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:32.694782 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325111a-261f-4985-91cb-ccc8fa84e7f6 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:32.696014 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:01:32.704991 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd46ee-4c99-4641-a2b7-24e0f8f67229 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:32.705564 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd46ee-4c99-4641-a2b7-24e0f8f67229 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:32.706003 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd46ee-4c99-4641-a2b7-24e0f8f67229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:32.792062 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-64fd46ee-4c99-4641-a2b7-24e0f8f67229 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:32.793360 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-64fd46ee-4c99-4641-a2b7-24e0f8f67229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:32.913587 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:01:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:33.065592 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d325111a-261f-4985-91cb-ccc8fa84e7f6 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:33.067181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d325111a-261f-4985-91cb-ccc8fa84e7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:33.172155 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:33.242162 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d325111a-261f-4985-91cb-ccc8fa84e7f6 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:33.373757 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06836a81-ed59-496b-8492-19d5eb70704b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:33.374384 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06836a81-ed59-496b-8492-19d5eb70704b service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:33.374655 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06836a81-ed59-496b-8492-19d5eb70704b service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:33.374920 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06836a81-ed59-496b-8492-19d5eb70704b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:33.376610 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-06836a81-ed59-496b-8492-19d5eb70704b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:33.428753 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:33.472642 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06836a81-ed59-496b-8492-19d5eb70704b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:33.474274 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06836a81-ed59-496b-8492-19d5eb70704b service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:33.506608 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 718/1428] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:01:34.405412 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e8db2-eb4f-40be-8d61-2df1767cb8bf None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:34.406303 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e8db2-eb4f-40be-8d61-2df1767cb8bf None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:34.406495 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e8db2-eb4f-40be-8d61-2df1767cb8bf None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:34.408981 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:01:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:34.415832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640b319-9291-4508-a01f-4067948d8157 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:34.416673 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640b319-9291-4508-a01f-4067948d8157 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:34.417018 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640b319-9291-4508-a01f-4067948d8157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:34.511977 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d640b319-9291-4508-a01f-4067948d8157 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:34.513260 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d640b319-9291-4508-a01f-4067948d8157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:34.630913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:01:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:34.809214 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e8db2-eb4f-40be-8d61-2df1767cb8bf None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:34.811388 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e42e8db2-eb4f-40be-8d61-2df1767cb8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:34.873533 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:35.043548 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e8db2-eb4f-40be-8d61-2df1767cb8bf None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:35.222957 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94237a92-7448-401a-83f5-3e5bc8206a9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:35.224342 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94237a92-7448-401a-83f5-3e5bc8206a9d service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:35.224854 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94237a92-7448-401a-83f5-3e5bc8206a9d service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:35.225351 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94237a92-7448-401a-83f5-3e5bc8206a9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:35.229239 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-94237a92-7448-401a-83f5-3e5bc8206a9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:35.243138 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:35.359439 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94237a92-7448-401a-83f5-3e5bc8206a9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:35.361265 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94237a92-7448-401a-83f5-3e5bc8206a9d service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:35.386629 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 712/1431] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:01:36.230131 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60524b0-23ea-430e-b9b0-5e60d4e5ae29 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:36.230866 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60524b0-23ea-430e-b9b0-5e60d4e5ae29 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:36.230866 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60524b0-23ea-430e-b9b0-5e60d4e5ae29 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:36.232728 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:01:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:36.238637 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a99f5-a482-4795-a6ea-6d31537e1a2b None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:36.239425 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a99f5-a482-4795-a6ea-6d31537e1a2b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:36.239584 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a99f5-a482-4795-a6ea-6d31537e1a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:36.324799 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-397a99f5-a482-4795-a6ea-6d31537e1a2b None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:36.326060 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-397a99f5-a482-4795-a6ea-6d31537e1a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:36.399781 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:36.436329 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:01:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:36.597649 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60524b0-23ea-430e-b9b0-5e60d4e5ae29 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:36.600304 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a60524b0-23ea-430e-b9b0-5e60d4e5ae29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:36.621249 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:36.765466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60524b0-23ea-430e-b9b0-5e60d4e5ae29 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:36.862624 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:36.891579 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208b0f6-b52d-4016-a156-81f4d5b473d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:36.892217 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208b0f6-b52d-4016-a156-81f4d5b473d4 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:36.892529 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208b0f6-b52d-4016-a156-81f4d5b473d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:36.892802 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208b0f6-b52d-4016-a156-81f4d5b473d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:36.894434 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0208b0f6-b52d-4016-a156-81f4d5b473d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:36.994473 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208b0f6-b52d-4016-a156-81f4d5b473d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:36.996158 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208b0f6-b52d-4016-a156-81f4d5b473d4 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:37.015626 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 721/1434] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:01:37.845897 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c02c05-064b-4d02-8d4f-54e86dad231e None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:37.846579 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c02c05-064b-4d02-8d4f-54e86dad231e None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:37.846579 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c02c05-064b-4d02-8d4f-54e86dad231e None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:37.847808 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:37.853246 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf46f0-0a5c-45eb-ac5c-355c5370c41f None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:37.853893 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf46f0-0a5c-45eb-ac5c-355c5370c41f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:37.853893 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf46f0-0a5c-45eb-ac5c-355c5370c41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:37.938556 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-5daf46f0-0a5c-45eb-ac5c-355c5370c41f None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:37.940016 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5daf46f0-0a5c-45eb-ac5c-355c5370c41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:37.986590 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:38.045847 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:01:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:38.200184 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c02c05-064b-4d02-8d4f-54e86dad231e None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:38.201751 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-89c02c05-064b-4d02-8d4f-54e86dad231e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:38.380593 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c02c05-064b-4d02-8d4f-54e86dad231e None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:38.445588 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:38.520894 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92470293-1986-41cd-a39b-06616062c84c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:38.522392 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92470293-1986-41cd-a39b-06616062c84c service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:38.524022 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92470293-1986-41cd-a39b-06616062c84c service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:38.524022 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92470293-1986-41cd-a39b-06616062c84c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:38.524874 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-92470293-1986-41cd-a39b-06616062c84c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:38.658237 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92470293-1986-41cd-a39b-06616062c84c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:38.660369 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92470293-1986-41cd-a39b-06616062c84c service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:38.681886 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 715/1437] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:01:39.534810 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bafd32-c99f-449e-88bf-a508efa221d8 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:39.535563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bafd32-c99f-449e-88bf-a508efa221d8 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:39.535563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bafd32-c99f-449e-88bf-a508efa221d8 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:39.536948 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:39.543328 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f688104-f55e-4416-8428-31dc29f3023e None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:39.544158 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f688104-f55e-4416-8428-31dc29f3023e None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:39.544158 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f688104-f55e-4416-8428-31dc29f3023e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:39.629276 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-6f688104-f55e-4416-8428-31dc29f3023e None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:39.633265 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-6f688104-f55e-4416-8428-31dc29f3023e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:39.636053 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:39.764585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:01:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:40.014627 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bafd32-c99f-449e-88bf-a508efa221d8 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:40.016246 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-08bafd32-c99f-449e-88bf-a508efa221d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:40.189826 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bafd32-c99f-449e-88bf-a508efa221d8 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:40.221045 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:40.316918 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a73bad-b968-4448-8915-9083507b9920 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:40.317591 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a73bad-b968-4448-8915-9083507b9920 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:40.317851 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a73bad-b968-4448-8915-9083507b9920 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:40.318122 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a73bad-b968-4448-8915-9083507b9920 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:40.319843 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-67a73bad-b968-4448-8915-9083507b9920 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:40.422603 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a73bad-b968-4448-8915-9083507b9920 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:40.424308 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a73bad-b968-4448-8915-9083507b9920 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:40.444858 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 724/1440] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:01:42.418568 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d758c7-257e-4984-b77d-471d7dd94e2a None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:42.419201 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d758c7-257e-4984-b77d-471d7dd94e2a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:42.419201 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d758c7-257e-4984-b77d-471d7dd94e2a None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:42.420639 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:42.426718 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e88f26-f520-40a4-8bc8-8f570163b3ca None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:42.427307 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e88f26-f520-40a4-8bc8-8f570163b3ca None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:42.427307 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e88f26-f520-40a4-8bc8-8f570163b3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:42.471199 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:42.507929 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b0e88f26-f520-40a4-8bc8-8f570163b3ca None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:42.509245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b0e88f26-f520-40a4-8bc8-8f570163b3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:42.629453 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:01:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:42.673399 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d758c7-257e-4984-b77d-471d7dd94e2a None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:42.675046 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c4d758c7-257e-4984-b77d-471d7dd94e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:42.796130 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d758c7-257e-4984-b77d-471d7dd94e2a None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:42.833793 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:42.884910 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cad44d-8168-4ede-8ce5-f0a25427bb32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:42.885564 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cad44d-8168-4ede-8ce5-f0a25427bb32 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:42.885844 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cad44d-8168-4ede-8ce5-f0a25427bb32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:42.886115 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cad44d-8168-4ede-8ce5-f0a25427bb32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:42.887899 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-86cad44d-8168-4ede-8ce5-f0a25427bb32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:43.023704 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cad44d-8168-4ede-8ce5-f0a25427bb32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:43.025403 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cad44d-8168-4ede-8ce5-f0a25427bb32 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:43.046837 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 718/1443] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:01:45.284603 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0efa6bb-4c73-43c1-a920-73baaed769ce None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:45.285180 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0efa6bb-4c73-43c1-a920-73baaed769ce None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:45.285180 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0efa6bb-4c73-43c1-a920-73baaed769ce None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:45.286460 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:01:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:45.293367 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df82339-1f7f-4056-97e2-0168c104ac7c None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:45.293919 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df82339-1f7f-4056-97e2-0168c104ac7c None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:45.293919 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df82339-1f7f-4056-97e2-0168c104ac7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:45.372564 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-7df82339-1f7f-4056-97e2-0168c104ac7c None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:45.373974 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7df82339-1f7f-4056-97e2-0168c104ac7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:45.388327 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:45.478516 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:01:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:45.517366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0efa6bb-4c73-43c1-a920-73baaed769ce None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:45.518962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d0efa6bb-4c73-43c1-a920-73baaed769ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:45.641405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0efa6bb-4c73-43c1-a920-73baaed769ce None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:45.657033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:45.728474 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f6e85b-c92b-47e0-a705-4bad09f16e46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:45.729104 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6e85b-c92b-47e0-a705-4bad09f16e46 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:45.729377 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6e85b-c92b-47e0-a705-4bad09f16e46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:45.729646 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6e85b-c92b-47e0-a705-4bad09f16e46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:45.731318 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-75f6e85b-c92b-47e0-a705-4bad09f16e46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:45.854292 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f6e85b-c92b-47e0-a705-4bad09f16e46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:45.855666 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f6e85b-c92b-47e0-a705-4bad09f16e46 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:45.875943 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 727/1446] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:01:48.264783 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0077a-2620-42bb-904b-aefb82ed98c8 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:48.265390 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0077a-2620-42bb-904b-aefb82ed98c8 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:48.265390 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0077a-2620-42bb-904b-aefb82ed98c8 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:48.266935 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:01:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:48.273378 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad96fea-7b7c-4727-b04b-80320121c54a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:48.273893 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad96fea-7b7c-4727-b04b-80320121c54a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:48.274115 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad96fea-7b7c-4727-b04b-80320121c54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:48.338322 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:48.363002 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-9ad96fea-7b7c-4727-b04b-80320121c54a None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:48.364256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9ad96fea-7b7c-4727-b04b-80320121c54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:48.479122 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:01:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:48.519476 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f0077a-2620-42bb-904b-aefb82ed98c8 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:48.521038 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-01f0077a-2620-42bb-904b-aefb82ed98c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:48.644200 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f0077a-2620-42bb-904b-aefb82ed98c8 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:48.694511 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:48.735887 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2cf1ae-d784-4a8d-8b35-d868a49887a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:48.736721 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2cf1ae-d784-4a8d-8b35-d868a49887a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:48.736998 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2cf1ae-d784-4a8d-8b35-d868a49887a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:48.737266 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2cf1ae-d784-4a8d-8b35-d868a49887a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:48.738944 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7a2cf1ae-d784-4a8d-8b35-d868a49887a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:48.863553 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2cf1ae-d784-4a8d-8b35-d868a49887a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:48.865023 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2cf1ae-d784-4a8d-8b35-d868a49887a9 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:48.890193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 721/1449] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:01:51.167822 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556c980-89dc-49b2-ba99-e1c702779fc8 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:51.168560 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556c980-89dc-49b2-ba99-e1c702779fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:51.168560 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556c980-89dc-49b2-ba99-e1c702779fc8 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:51.169854 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:01:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:51.176380 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93ba31-6bb5-4c6a-88b4-9c573ba0674e None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:51.177257 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93ba31-6bb5-4c6a-88b4-9c573ba0674e None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:51.177257 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93ba31-6bb5-4c6a-88b4-9c573ba0674e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:51.393241 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-4b93ba31-6bb5-4c6a-88b4-9c573ba0674e None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:51.394459 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-4b93ba31-6bb5-4c6a-88b4-9c573ba0674e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:51.416530 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:51.509991 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:51.552477 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556c980-89dc-49b2-ba99-e1c702779fc8 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:51.554939 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7556c980-89dc-49b2-ba99-e1c702779fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:51.682719 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556c980-89dc-49b2-ba99-e1c702779fc8 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:51.705941 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:51.767670 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3bf116-b5e0-4c94-96d1-f883ba26b125 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:51.768321 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bf116-b5e0-4c94-96d1-f883ba26b125 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:51.768600 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bf116-b5e0-4c94-96d1-f883ba26b125 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:51.768869 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bf116-b5e0-4c94-96d1-f883ba26b125 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:51.770510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4d3bf116-b5e0-4c94-96d1-f883ba26b125 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:51.896043 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3bf116-b5e0-4c94-96d1-f883ba26b125 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:51.897535 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3bf116-b5e0-4c94-96d1-f883ba26b125 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:51.917226 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 730/1452] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:01:54.157788 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb50c6e-a468-4756-8db3-cfa482cdae03 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:54.158392 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb50c6e-a468-4756-8db3-cfa482cdae03 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:54.158392 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb50c6e-a468-4756-8db3-cfa482cdae03 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:54.159714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:01:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:54.166306 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb9e10-c896-476c-b541-fa518b3c2901 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:54.166925 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb9e10-c896-476c-b541-fa518b3c2901 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:54.166925 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb9e10-c896-476c-b541-fa518b3c2901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:54.266172 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:54.269565 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c7bb9e10-c896-476c-b541-fa518b3c2901 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:54.271335 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c7bb9e10-c896-476c-b541-fa518b3c2901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:54.383202 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:01:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:01:54.423151 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb50c6e-a468-4756-8db3-cfa482cdae03 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:54.424733 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7bb50c6e-a468-4756-8db3-cfa482cdae03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:54.534848 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb50c6e-a468-4756-8db3-cfa482cdae03 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:54.599584 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:54.626549 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b18d4c-e1a7-4e31-89e8-f97a1914ad31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:54.627185 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b18d4c-e1a7-4e31-89e8-f97a1914ad31 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:54.627464 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b18d4c-e1a7-4e31-89e8-f97a1914ad31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:54.627735 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b18d4c-e1a7-4e31-89e8-f97a1914ad31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:54.629437 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-38b18d4c-e1a7-4e31-89e8-f97a1914ad31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:54.773752 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b18d4c-e1a7-4e31-89e8-f97a1914ad31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:54.775425 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b18d4c-e1a7-4e31-89e8-f97a1914ad31 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:54.796576 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 724/1455] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:01:56.908881 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6291b-a7fe-4bcf-9ffb-522b4ecff3d2 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:56.909467 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6291b-a7fe-4bcf-9ffb-522b4ecff3d2 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:56.909467 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6291b-a7fe-4bcf-9ffb-522b4ecff3d2 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:56.910714 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:56.916291 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bd88a-355f-4cbe-8c2b-027dd21ea972 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:56.916840 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bd88a-355f-4cbe-8c2b-027dd21ea972 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:56.916840 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bd88a-355f-4cbe-8c2b-027dd21ea972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:56.990699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-722bd88a-355f-4cbe-8c2b-027dd21ea972 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:56.991377 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-722bd88a-355f-4cbe-8c2b-027dd21ea972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:57.024511 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:57.103350 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:01:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:57.142943 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c6291b-a7fe-4bcf-9ffb-522b4ecff3d2 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:57.145003 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-40c6291b-a7fe-4bcf-9ffb-522b4ecff3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:57.258103 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c6291b-a7fe-4bcf-9ffb-522b4ecff3d2 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:57.296523 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:57.345896 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ce06f-ec82-4f44-854c-66a92bc770d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:57.346526 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ce06f-ec82-4f44-854c-66a92bc770d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:57.346789 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ce06f-ec82-4f44-854c-66a92bc770d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:57.347061 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ce06f-ec82-4f44-854c-66a92bc770d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:57.348787 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-616ce06f-ec82-4f44-854c-66a92bc770d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:57.470629 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ce06f-ec82-4f44-854c-66a92bc770d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:57.472110 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ce06f-ec82-4f44-854c-66a92bc770d7 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:57.493861 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 733/1458] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:01:59.595342 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee6851-d117-465c-95de-c4ca29cc7044 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:59.595949 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee6851-d117-465c-95de-c4ca29cc7044 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:59.595949 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee6851-d117-465c-95de-c4ca29cc7044 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:59.597346 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:59.603297 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f95b4-cf11-4376-8035-989b0066b015 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:59.603797 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f95b4-cf11-4376-8035-989b0066b015 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:59.603797 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f95b4-cf11-4376-8035-989b0066b015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:59.688482 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-217f95b4-cf11-4376-8035-989b0066b015 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:59.689749 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-217f95b4-cf11-4376-8035-989b0066b015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:59.701255 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:59.810040 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:01:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:59.870033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee6851-d117-465c-95de-c4ca29cc7044 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:59.872726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3eee6851-d117-465c-95de-c4ca29cc7044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:00.010078 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee6851-d117-465c-95de-c4ca29cc7044 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:02:00.075955 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:00.095438 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c0895-7cea-4fee-951f-d50143882134 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:02:00.096077 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c0895-7cea-4fee-951f-d50143882134 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:00.096392 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c0895-7cea-4fee-951f-d50143882134 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:00.096685 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c0895-7cea-4fee-951f-d50143882134 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:00.098367 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a56c0895-7cea-4fee-951f-d50143882134 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:00.215329 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c0895-7cea-4fee-951f-d50143882134 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:02:00.216811 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c0895-7cea-4fee-951f-d50143882134 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:02:00.242740 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 727/1461] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:02:02.358676 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad7952-4a09-41aa-bd9b-6d61d067269a None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:02.359344 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad7952-4a09-41aa-bd9b-6d61d067269a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:02.359344 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad7952-4a09-41aa-bd9b-6d61d067269a None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:02.360750 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:02:02.366950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a766bc9-c657-45bc-8495-e48ea0f4dc30 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:02.367685 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a766bc9-c657-45bc-8495-e48ea0f4dc30 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:02.367685 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a766bc9-c657-45bc-8495-e48ea0f4dc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:02.443842 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-4a766bc9-c657-45bc-8495-e48ea0f4dc30 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:02:02.445137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-4a766bc9-c657-45bc-8495-e48ea0f4dc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:02.493750 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:02.564548 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:02:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:02:02.602586 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad7952-4a09-41aa-bd9b-6d61d067269a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:02:02.604284 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b3ad7952-4a09-41aa-bd9b-6d61d067269a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:02.722836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad7952-4a09-41aa-bd9b-6d61d067269a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:02:02.762864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:02.801864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa6b45-7d3a-40ce-970c-7dcc2062e420 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:02:02.802492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa6b45-7d3a-40ce-970c-7dcc2062e420 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:02.802778 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa6b45-7d3a-40ce-970c-7dcc2062e420 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:02.803178 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa6b45-7d3a-40ce-970c-7dcc2062e420 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:02.804870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-86aa6b45-7d3a-40ce-970c-7dcc2062e420 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:02.925171 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aa6b45-7d3a-40ce-970c-7dcc2062e420 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:02:02.926628 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aa6b45-7d3a-40ce-970c-7dcc2062e420 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:02:02.947047 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 736/1464] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:02:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:02:04.990589 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a168b98-2d8c-4a86-9532-1a5cf94ffab1 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:04.991197 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a168b98-2d8c-4a86-9532-1a5cf94ffab1 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:04.991197 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a168b98-2d8c-4a86-9532-1a5cf94ffab1 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:04.992534 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:02:04.999237 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e76fb14-c704-45af-88f4-63a2c2004171 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:04.999728 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e76fb14-c704-45af-88f4-63a2c2004171 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:04.999826 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e76fb14-c704-45af-88f4-63a2c2004171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:05.076635 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8e76fb14-c704-45af-88f4-63a2c2004171 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:02:05.077873 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8e76fb14-c704-45af-88f4-63a2c2004171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:05.099388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:05.197225 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:02:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:02:05.235858 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a168b98-2d8c-4a86-9532-1a5cf94ffab1 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:02:05.237551 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-2a168b98-2d8c-4a86-9532-1a5cf94ffab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:05.361688 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a168b98-2d8c-4a86-9532-1a5cf94ffab1 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:02:05.438354 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:05.447408 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e08e29-f079-4032-a877-3fbec38ee73b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:02:05.447408 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e08e29-f079-4032-a877-3fbec38ee73b service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:05.447408 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e08e29-f079-4032-a877-3fbec38ee73b service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:05.447408 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e08e29-f079-4032-a877-3fbec38ee73b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:05.449558 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-01e08e29-f079-4032-a877-3fbec38ee73b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:05.581795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e08e29-f079-4032-a877-3fbec38ee73b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:02:05.584048 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e08e29-f079-4032-a877-3fbec38ee73b service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:02:05.608294 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 730/1467] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:02:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:02:07.652592 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25212097-8c9f-441c-a532-5a7123ef6443 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:07.653225 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25212097-8c9f-441c-a532-5a7123ef6443 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:07.653225 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25212097-8c9f-441c-a532-5a7123ef6443 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:07.654789 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:02:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:02:07.660707 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0012bb-4748-49ec-b32d-947620e64519 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:07.661250 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0012bb-4748-49ec-b32d-947620e64519 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:07.661250 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0012bb-4748-49ec-b32d-947620e64519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:07.731643 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-ae0012bb-4748-49ec-b32d-947620e64519 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:02:07.732931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ae0012bb-4748-49ec-b32d-947620e64519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:07.783261 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:07.832963 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:02:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:02:07.870677 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25212097-8c9f-441c-a532-5a7123ef6443 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:02:07.872249 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-25212097-8c9f-441c-a532-5a7123ef6443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:07.980866 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25212097-8c9f-441c-a532-5a7123ef6443 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:02:08.033268 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:08.063063 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a19f6-b865-4d52-ad9a-c954dd268c21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:02:08.063692 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a19f6-b865-4d52-ad9a-c954dd268c21 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:08.063991 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a19f6-b865-4d52-ad9a-c954dd268c21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:08.064292 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a19f6-b865-4d52-ad9a-c954dd268c21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:08.065924 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fe1a19f6-b865-4d52-ad9a-c954dd268c21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:08.187549 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a19f6-b865-4d52-ad9a-c954dd268c21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:02:08.188972 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a19f6-b865-4d52-ad9a-c954dd268c21 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:02:08.208973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 739/1470] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:02:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:02:17.830917 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=22 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:18.206628 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=66 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:27.958673 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8f62c-2b96-4e91-9bdc-8869097a70a6 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:27.959238 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8f62c-2b96-4e91-9bdc-8869097a70a6 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:27.959238 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8f62c-2b96-4e91-9bdc-8869097a70a6 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:27.960609 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 732/1471] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 14:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:02:27.966451 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146f1c9-bcad-485e-8738-a9620f4fdda8 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:27.966971 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146f1c9-bcad-485e-8738-a9620f4fdda8 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:27.966971 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146f1c9-bcad-485e-8738-a9620f4fdda8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:28.037383 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-1146f1c9-bcad-485e-8738-a9620f4fdda8 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:02:28.038595 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1146f1c9-bcad-485e-8738-a9620f4fdda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:28.140056 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 740/1472] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 14:02:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:02:28.232066 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16d48b-fea7-4936-a7a2-2f51ed087701 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:28.232393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16d48b-fea7-4936-a7a2-2f51ed087701 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:28.232669 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16d48b-fea7-4936-a7a2-2f51ed087701 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:28.234045 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 733/1473] 10.222.0.45 () {54 vars in 966 bytes} [Mon Aug 1 14:02:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:02:28.239208 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08375e-3571-4e7f-8711-2ac04c8facff None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:28.239498 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08375e-3571-4e7f-8711-2ac04c8facff None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:28.239772 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08375e-3571-4e7f-8711-2ac04c8facff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:28.321981 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-fa08375e-3571-4e7f-8711-2ac04c8facff None None] MFA Rules not processed for user `e7317c8829cb4fa1b30711f1db575cab`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:02:28.323229 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fa08375e-3571-4e7f-8711-2ac04c8facff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:28.346210 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:28.413346 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 741/1474] 10.222.0.45 () {58 vars in 1063 bytes} [Mon Aug 1 14:02:28 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:02:28.421698 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa16d48b-fea7-4936-a7a2-2f51ed087701 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:02:28.423583 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-fa16d48b-fea7-4936-a7a2-2f51ed087701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:28.550990 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa16d48b-fea7-4936-a7a2-2f51ed087701 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:02:28.629517 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7942fd6-ecd6-4609-b773-a5f2d24ba4f9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7317c8829cb4fa1b30711f1db575cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:02:28.630141 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7942fd6-ecd6-4609-b773-a5f2d24ba4f9 service glance] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:28.630414 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7942fd6-ecd6-4609-b773-a5f2d24ba4f9 service glance] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:28.630677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7942fd6-ecd6-4609-b773-a5f2d24ba4f9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:28.632353 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c7942fd6-ecd6-4609-b773-a5f2d24ba4f9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:28.689822 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:28.749568 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7942fd6-ecd6-4609-b773-a5f2d24ba4f9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:02:28.751027 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7942fd6-ecd6-4609-b773-a5f2d24ba4f9 service glance] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:02:28.771030 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 734/1475] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:02:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:02:30.909184 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c25d5-2600-404d-9b72-c5bba497d640 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:30.909768 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c25d5-2600-404d-9b72-c5bba497d640 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:30.909768 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c25d5-2600-404d-9b72-c5bba497d640 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:30.911088 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 742/1476] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 14:02:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:02:30.916776 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d7ddc-02a7-4f6c-9875-94232586f52b None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:30.917323 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d7ddc-02a7-4f6c-9875-94232586f52b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:30.917323 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d7ddc-02a7-4f6c-9875-94232586f52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:30.995135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-897d7ddc-02a7-4f6c-9875-94232586f52b None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:02:30.996430 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-897d7ddc-02a7-4f6c-9875-94232586f52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:31.102032 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 735/1477] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 14:02:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:02:31.125672 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c25d5-2600-404d-9b72-c5bba497d640 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:02:31.127232 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-901c25d5-2600-404d-9b72-c5bba497d640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:31.270996 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c25d5-2600-404d-9b72-c5bba497d640 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:02:31.335744 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:31.391190 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2bc1c0-8faf-4552-9be7-f830bf97b750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:02:31.391856 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bc1c0-8faf-4552-9be7-f830bf97b750 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:31.392134 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bc1c0-8faf-4552-9be7-f830bf97b750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:31.392440 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bc1c0-8faf-4552-9be7-f830bf97b750 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:31.393138 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2bc1c0-8faf-4552-9be7-f830bf97b750 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:02:31.394919 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: WARNING py.warnings [None req-5b2bc1c0-8faf-4552-9be7-f830bf97b750 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:02:31.394919 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: warnings.warn(msg) Aug 01 14:02:31.394919 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: Aug 01 14:02:31.396058 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2bc1c0-8faf-4552-9be7-f830bf97b750 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:02:31.416743 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 743/1478] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 14:02:31 2022] GET /identity/v3/projects => generated 2553 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:02:33.489429 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e29735-d002-4a8f-8c92-325f072127d8 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:33.490073 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e29735-d002-4a8f-8c92-325f072127d8 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:33.490073 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e29735-d002-4a8f-8c92-325f072127d8 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:33.491445 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 736/1479] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 14:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:02:33.497359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68d2b1-bc99-4765-a6a8-5551bb107c86 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:33.497898 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68d2b1-bc99-4765-a6a8-5551bb107c86 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:33.497898 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68d2b1-bc99-4765-a6a8-5551bb107c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:33.569622 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-4e68d2b1-bc99-4765-a6a8-5551bb107c86 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:02:33.570917 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4e68d2b1-bc99-4765-a6a8-5551bb107c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:33.672912 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 744/1480] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 14:02:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:02:33.716677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e29735-d002-4a8f-8c92-325f072127d8 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:02:33.718264 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-46e29735-d002-4a8f-8c92-325f072127d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:33.729657 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:33.885757 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e29735-d002-4a8f-8c92-325f072127d8 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:02:33.973505 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:34.007386 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07224672-b78d-4544-91a7-acd83a83c2d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:02:34.007386 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07224672-b78d-4544-91a7-acd83a83c2d1 service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:34.008478 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07224672-b78d-4544-91a7-acd83a83c2d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:34.008828 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07224672-b78d-4544-91a7-acd83a83c2d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:34.010281 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-07224672-b78d-4544-91a7-acd83a83c2d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:34.110065 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07224672-b78d-4544-91a7-acd83a83c2d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:02:34.111809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07224672-b78d-4544-91a7-acd83a83c2d1 service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:02:34.131536 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 737/1481] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:02:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:02:36.348880 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273bc6e8-496f-44c4-9c63-82364a522538 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:36.349521 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273bc6e8-496f-44c4-9c63-82364a522538 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:36.349521 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273bc6e8-496f-44c4-9c63-82364a522538 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:36.350812 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 745/1482] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 14:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:02:36.356507 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc0573-d683-427f-acc8-1926d9eb324c None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:36.356994 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc0573-d683-427f-acc8-1926d9eb324c None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:36.357100 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc0573-d683-427f-acc8-1926d9eb324c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:36.430616 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-3ddc0573-d683-427f-acc8-1926d9eb324c None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:02:36.431867 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3ddc0573-d683-427f-acc8-1926d9eb324c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:36.483886 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:36.553065 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 738/1483] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 14:02:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:02:36.590560 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273bc6e8-496f-44c4-9c63-82364a522538 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:02:36.592175 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-273bc6e8-496f-44c4-9c63-82364a522538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:36.619230 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:36.718274 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273bc6e8-496f-44c4-9c63-82364a522538 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:02:36.797184 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad69bf-bcfb-4af1-9c1a-5c7db9670098 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:02:36.797817 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad69bf-bcfb-4af1-9c1a-5c7db9670098 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:36.798100 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad69bf-bcfb-4af1-9c1a-5c7db9670098 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:36.798374 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad69bf-bcfb-4af1-9c1a-5c7db9670098 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:36.800024 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-59ad69bf-bcfb-4af1-9c1a-5c7db9670098 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:36.878271 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:36.927858 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad69bf-bcfb-4af1-9c1a-5c7db9670098 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:02:36.929613 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad69bf-bcfb-4af1-9c1a-5c7db9670098 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:02:36.954448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 746/1484] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:02:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:02:39.030329 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82973e95-0261-42aa-914b-c29896643d94 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:39.030860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82973e95-0261-42aa-914b-c29896643d94 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:39.030860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82973e95-0261-42aa-914b-c29896643d94 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:39.032236 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 739/1485] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 14:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:02:39.037881 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccabc3-dea5-4029-982f-b35f8029597d None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:39.038467 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccabc3-dea5-4029-982f-b35f8029597d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:39.038467 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccabc3-dea5-4029-982f-b35f8029597d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:39.111267 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-12ccabc3-dea5-4029-982f-b35f8029597d None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:02:39.112447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-12ccabc3-dea5-4029-982f-b35f8029597d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:39.213344 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 747/1486] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 14:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:02:39.251998 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82973e95-0261-42aa-914b-c29896643d94 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:02:39.253620 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-82973e95-0261-42aa-914b-c29896643d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:39.370599 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82973e95-0261-42aa-914b-c29896643d94 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:02:39.450035 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:39.451845 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd37bc-3d9f-4bfb-96ab-02656b78c2f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:02:39.452517 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd37bc-3d9f-4bfb-96ab-02656b78c2f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:39.452804 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd37bc-3d9f-4bfb-96ab-02656b78c2f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:39.453095 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd37bc-3d9f-4bfb-96ab-02656b78c2f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:39.455414 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d6fd37bc-3d9f-4bfb-96ab-02656b78c2f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:39.578254 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd37bc-3d9f-4bfb-96ab-02656b78c2f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:02:39.579690 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd37bc-3d9f-4bfb-96ab-02656b78c2f2 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:02:39.599107 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 740/1487] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:02:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:02:41.817555 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d4e50-21a5-4f37-a689-1225623f9fca None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:41.818358 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d4e50-21a5-4f37-a689-1225623f9fca None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:41.818358 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d4e50-21a5-4f37-a689-1225623f9fca None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:41.819401 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 748/1488] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 14:02:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:02:41.825412 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af79fff-fbc9-4e0f-b091-2ebd5efecfd4 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:41.825927 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af79fff-fbc9-4e0f-b091-2ebd5efecfd4 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:41.825927 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af79fff-fbc9-4e0f-b091-2ebd5efecfd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:41.903022 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-5af79fff-fbc9-4e0f-b091-2ebd5efecfd4 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:02:41.904292 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5af79fff-fbc9-4e0f-b091-2ebd5efecfd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:41.962258 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:42.010908 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 741/1489] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 14:02:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:02:42.052566 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d4e50-21a5-4f37-a689-1225623f9fca None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:02:42.054119 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7e6d4e50-21a5-4f37-a689-1225623f9fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:42.094080 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:42.173366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d4e50-21a5-4f37-a689-1225623f9fca None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:02:42.253733 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d63795-9342-454a-844f-6dc70958008d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:02:42.254371 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d63795-9342-454a-844f-6dc70958008d service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:42.254653 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d63795-9342-454a-844f-6dc70958008d service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:42.254958 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d63795-9342-454a-844f-6dc70958008d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:42.256681 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c9d63795-9342-454a-844f-6dc70958008d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:42.345231 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:42.378416 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d63795-9342-454a-844f-6dc70958008d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:02:42.379906 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d63795-9342-454a-844f-6dc70958008d service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:02:42.400922 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 749/1490] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:02:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:02:44.667960 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799e218-0cd1-406b-8ead-d6649a0dcd59 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:44.668591 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799e218-0cd1-406b-8ead-d6649a0dcd59 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:44.668591 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799e218-0cd1-406b-8ead-d6649a0dcd59 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:44.670014 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 742/1491] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 14:02:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:02:44.675908 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d2e3e-7b70-4e2c-919a-74648d7f4b49 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:44.676447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d2e3e-7b70-4e2c-919a-74648d7f4b49 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:44.676566 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d2e3e-7b70-4e2c-919a-74648d7f4b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:44.748055 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-700d2e3e-7b70-4e2c-919a-74648d7f4b49 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:02:44.749343 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-700d2e3e-7b70-4e2c-919a-74648d7f4b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:44.855930 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 750/1492] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 14:02:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:02:44.896398 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6799e218-0cd1-406b-8ead-d6649a0dcd59 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:02:44.898332 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-6799e218-0cd1-406b-8ead-d6649a0dcd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:45.012537 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6799e218-0cd1-406b-8ead-d6649a0dcd59 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:02:45.094270 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a437a32-cf84-49df-ac8f-05eba7c0ae5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:02:45.094909 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a437a32-cf84-49df-ac8f-05eba7c0ae5e service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:45.095186 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a437a32-cf84-49df-ac8f-05eba7c0ae5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:45.095460 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a437a32-cf84-49df-ac8f-05eba7c0ae5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:45.097146 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0a437a32-cf84-49df-ac8f-05eba7c0ae5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:45.107584 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:45.234622 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a437a32-cf84-49df-ac8f-05eba7c0ae5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:02:45.236462 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a437a32-cf84-49df-ac8f-05eba7c0ae5e service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:02:45.259119 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 743/1493] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:02:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:02:54.853927 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=82 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:55.262797 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=50 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:24.103057 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193a569-535b-48fb-b92d-2494fb21a1e2 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:24.103693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193a569-535b-48fb-b92d-2494fb21a1e2 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:24.103866 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193a569-535b-48fb-b92d-2494fb21a1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:24.226437 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-e193a569-535b-48fb-b92d-2494fb21a1e2 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:04:24.228220 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e193a569-535b-48fb-b92d-2494fb21a1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:04:24.344514 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 751/1494] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:04:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:04:24.355892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc542bb2-99fc-4167-b1b6-99a8a9405e48 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:24.356743 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc542bb2-99fc-4167-b1b6-99a8a9405e48 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:24.356743 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc542bb2-99fc-4167-b1b6-99a8a9405e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:24.473549 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-bc542bb2-99fc-4167-b1b6-99a8a9405e48 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:04:24.475369 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-bc542bb2-99fc-4167-b1b6-99a8a9405e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:04:24.615395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 744/1495] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:04:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:04:24.620436 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e193a569-535b-48fb-b92d-2494fb21a1e2 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:24.621868 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e193a569-535b-48fb-b92d-2494fb21a1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:04:24.634496 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e193a569-535b-48fb-b92d-2494fb21a1e2 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:24.741480 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:24.781441 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f8796-18a5-4ef9-88a1-8a971f2e27bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:24.782015 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f8796-18a5-4ef9-88a1-8a971f2e27bd admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:24.782288 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f8796-18a5-4ef9-88a1-8a971f2e27bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:24.782610 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f8796-18a5-4ef9-88a1-8a971f2e27bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:24.783294 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244f8796-18a5-4ef9-88a1-8a971f2e27bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:04:24.783542 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244f8796-18a5-4ef9-88a1-8a971f2e27bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:24.784598 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: WARNING py.warnings [None req-244f8796-18a5-4ef9-88a1-8a971f2e27bd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:04:24.784598 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: warnings.warn(msg) Aug 01 14:04:24.784598 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: Aug 01 14:04:24.785539 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244f8796-18a5-4ef9-88a1-8a971f2e27bd admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:24.800067 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 752/1496] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:04:24.808922 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc542bb2-99fc-4167-b1b6-99a8a9405e48 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:24.819130 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc542bb2-99fc-4167-b1b6-99a8a9405e48 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:24.926146 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:24.987401 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7618dd7d-1f7b-4ec2-a6ab-541b8a40789e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:24.988517 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7618dd7d-1f7b-4ec2-a6ab-541b8a40789e admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:24.989022 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7618dd7d-1f7b-4ec2-a6ab-541b8a40789e admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:24.989508 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7618dd7d-1f7b-4ec2-a6ab-541b8a40789e admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:24.990748 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7618dd7d-1f7b-4ec2-a6ab-541b8a40789e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:24.994069 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: WARNING py.warnings [None req-7618dd7d-1f7b-4ec2-a6ab-541b8a40789e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:04:24.994069 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: warnings.warn(msg) Aug 01 14:04:24.994069 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: Aug 01 14:04:24.996463 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7618dd7d-1f7b-4ec2-a6ab-541b8a40789e admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:25.084115 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 745/1497] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:04:24 2022] POST /identity/v3/projects => generated 379 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:04:25.093483 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f8796-18a5-4ef9-88a1-8a971f2e27bd admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:25.096011 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-244f8796-18a5-4ef9-88a1-8a971f2e27bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:04:25.150814 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f8796-18a5-4ef9-88a1-8a971f2e27bd admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:25.254016 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f8333d-07c6-4679-8740-eb3f300fb918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:25.254678 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8333d-07c6-4679-8740-eb3f300fb918 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:25.255058 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8333d-07c6-4679-8740-eb3f300fb918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:25.255367 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8333d-07c6-4679-8740-eb3f300fb918 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:25.256034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f8333d-07c6-4679-8740-eb3f300fb918 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:25.257131 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: WARNING py.warnings [None req-40f8333d-07c6-4679-8740-eb3f300fb918 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:04:25.257131 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: warnings.warn(msg) Aug 01 14:04:25.257131 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: Aug 01 14:04:25.258360 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f8333d-07c6-4679-8740-eb3f300fb918 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:25.303752 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 753/1498] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:04:25 2022] POST /identity/v3/users => generated 290 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:04:25.310739 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7618dd7d-1f7b-4ec2-a6ab-541b8a40789e admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:25.313266 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7618dd7d-1f7b-4ec2-a6ab-541b8a40789e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:04:25.381704 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7618dd7d-1f7b-4ec2-a6ab-541b8a40789e admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:25.496320 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:25.510728 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2505c01c-20a7-46cd-9d66-e27afda8f58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:25.511649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505c01c-20a7-46cd-9d66-e27afda8f58b admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:25.512030 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505c01c-20a7-46cd-9d66-e27afda8f58b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:25.512489 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505c01c-20a7-46cd-9d66-e27afda8f58b admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:25.513639 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2505c01c-20a7-46cd-9d66-e27afda8f58b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:25.515288 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: WARNING py.warnings [None req-2505c01c-20a7-46cd-9d66-e27afda8f58b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:04:25.515288 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: warnings.warn(msg) Aug 01 14:04:25.515288 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: Aug 01 14:04:25.516795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2505c01c-20a7-46cd-9d66-e27afda8f58b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:25.538495 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 746/1499] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:04:25.542966 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f8333d-07c6-4679-8740-eb3f300fb918 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:25.544665 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-40f8333d-07c6-4679-8740-eb3f300fb918 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:04:25.559022 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f8333d-07c6-4679-8740-eb3f300fb918 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:25.586477 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:25.667343 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b0bf9-2121-4d93-a9fe-e7ae06c43038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:25.668159 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b0bf9-2121-4d93-a9fe-e7ae06c43038 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:25.668562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b0bf9-2121-4d93-a9fe-e7ae06c43038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:25.668833 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b0bf9-2121-4d93-a9fe-e7ae06c43038 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:25.669534 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b0bf9-2121-4d93-a9fe-e7ae06c43038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:25.670596 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: WARNING py.warnings [None req-968b0bf9-2121-4d93-a9fe-e7ae06c43038 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:04:25.670596 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: warnings.warn(msg) Aug 01 14:04:25.670596 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: Aug 01 14:04:25.671559 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b0bf9-2121-4d93-a9fe-e7ae06c43038 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:25.685308 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 754/1500] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:04:25.689816 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2505c01c-20a7-46cd-9d66-e27afda8f58b admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:25.691450 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-2505c01c-20a7-46cd-9d66-e27afda8f58b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:04:25.729410 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2505c01c-20a7-46cd-9d66-e27afda8f58b admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:25.844937 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20bf3e-66c2-4a4a-8149-3cb885cf2f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:25.845523 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20bf3e-66c2-4a4a-8149-3cb885cf2f3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:25.845808 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20bf3e-66c2-4a4a-8149-3cb885cf2f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:25.846075 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20bf3e-66c2-4a4a-8149-3cb885cf2f3c admin admin] PATH_INFO: `/v3/projects/1bd0f694d4ea4dc4bc923e08c2e2b503/users/c075424c23764cdfaf513523c366324a/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:25.873671 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20bf3e-66c2-4a4a-8149-3cb885cf2f3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bd0f694d4ea4dc4bc923e08c2e2b503, user_id=c075424c23764cdfaf513523c366324a, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:25.874740 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: WARNING py.warnings [None req-5c20bf3e-66c2-4a4a-8149-3cb885cf2f3c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:04:25.874740 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: warnings.warn(msg) Aug 01 14:04:25.874740 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: Aug 01 14:04:25.878058 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20bf3e-66c2-4a4a-8149-3cb885cf2f3c admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:25.909211 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 747/1501] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:04:25 2022] PUT /identity/v3/projects/1bd0f694d4ea4dc4bc923e08c2e2b503/users/c075424c23764cdfaf513523c366324a/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:04:25.927637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d30fb2-fbea-4216-a821-d0aa100ca63d None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:25.928064 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d30fb2-fbea-4216-a821-d0aa100ca63d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:25.928534 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d30fb2-fbea-4216-a821-d0aa100ca63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:26.012838 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-97d30fb2-fbea-4216-a821-d0aa100ca63d None None] MFA Rules not processed for user `c075424c23764cdfaf513523c366324a`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:04:26.014950 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-97d30fb2-fbea-4216-a821-d0aa100ca63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:04:26.135478 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 755/1502] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:04:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:04:26.152236 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef76ff7-f094-48bc-b50d-81bd8707b4a3 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:26.152696 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef76ff7-f094-48bc-b50d-81bd8707b4a3 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:26.153108 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef76ff7-f094-48bc-b50d-81bd8707b4a3 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:26.155328 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 748/1503] 10.222.0.45 () {54 vars in 964 bytes} [Mon Aug 1 14:04:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:04:26.161013 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3c6dd-b002-4b54-a4e6-01abc71750f2 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:26.161568 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3c6dd-b002-4b54-a4e6-01abc71750f2 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:26.161568 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3c6dd-b002-4b54-a4e6-01abc71750f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:26.198138 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:26.275784 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-cee3c6dd-b002-4b54-a4e6-01abc71750f2 None None] MFA Rules not processed for user `8f4c3d46ac7243aeb722a4410f40d846`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:04:26.277649 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-cee3c6dd-b002-4b54-a4e6-01abc71750f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:04:26.412241 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 756/1504] 10.222.0.45 () {58 vars in 1061 bytes} [Mon Aug 1 14:04:26 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:04:26.420943 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef76ff7-f094-48bc-b50d-81bd8707b4a3 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:26.422655 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0ef76ff7-f094-48bc-b50d-81bd8707b4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:04:26.480622 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:26.569128 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef76ff7-f094-48bc-b50d-81bd8707b4a3 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:26.662937 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d72d5-e3da-48e0-ac0d-cd00ac2ea09f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:26.663861 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d72d5-e3da-48e0-ac0d-cd00ac2ea09f service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:26.664346 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d72d5-e3da-48e0-ac0d-cd00ac2ea09f service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:26.664763 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d72d5-e3da-48e0-ac0d-cd00ac2ea09f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:26.667369 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0e3d72d5-e3da-48e0-ac0d-cd00ac2ea09f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:04:26.810603 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d72d5-e3da-48e0-ac0d-cd00ac2ea09f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:26.812994 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d72d5-e3da-48e0-ac0d-cd00ac2ea09f service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:26.818661 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:26.842531 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 749/1505] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:04:26 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:04:26.894891 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ad9ac-b174-4802-ad80-1adf40a6d539 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:26.895333 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ad9ac-b174-4802-ad80-1adf40a6d539 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:26.895747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ad9ac-b174-4802-ad80-1adf40a6d539 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:26.898270 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 757/1506] 10.222.0.45 () {52 vars in 830 bytes} [Mon Aug 1 14:04:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:04:26.902752 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d72d5-e3da-48e0-ac0d-cd00ac2ea09f service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:26.904081 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0e3d72d5-e3da-48e0-ac0d-cd00ac2ea09f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:04:27.021950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d72d5-e3da-48e0-ac0d-cd00ac2ea09f service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:27.129610 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76bb542-fe7d-41c2-aa41-fba3ab0f9d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:27.130342 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76bb542-fe7d-41c2-aa41-fba3ab0f9d29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:27.130671 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76bb542-fe7d-41c2-aa41-fba3ab0f9d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:27.131000 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76bb542-fe7d-41c2-aa41-fba3ab0f9d29 admin admin] PATH_INFO: `/v3/users/c075424c23764cdfaf513523c366324a` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:27.145233 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76bb542-fe7d-41c2-aa41-fba3ab0f9d29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c075424c23764cdfaf513523c366324a)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:27.146185 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: WARNING py.warnings [None req-f76bb542-fe7d-41c2-aa41-fba3ab0f9d29 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:04:27.146185 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: warnings.warn(msg) Aug 01 14:04:27.146185 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: Aug 01 14:04:27.147430 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76bb542-fe7d-41c2-aa41-fba3ab0f9d29 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:27.158504 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:27.226491 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-f76bb542-fe7d-41c2-aa41-fba3ab0f9d29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c075424c23764cdfaf513523c366324a'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:04:27.227246 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-f76bb542-fe7d-41c2-aa41-fba3ab0f9d29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c075424c23764cdfaf513523c366324a'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:04:27.242428 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-f76bb542-fe7d-41c2-aa41-fba3ab0f9d29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c075424c23764cdfaf513523c366324a'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:04:27.257653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-f76bb542-fe7d-41c2-aa41-fba3ab0f9d29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c075424c23764cdfaf513523c366324a'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:04:27.282679 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-f76bb542-fe7d-41c2-aa41-fba3ab0f9d29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c075424c23764cdfaf513523c366324a'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:04:27.289275 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 750/1507] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:04:26 2022] DELETE /identity/v3/users/c075424c23764cdfaf513523c366324a => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:04:27.293970 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ca8f2-1ec5-4abe-b7d0-ab96f66b74ee None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:27.294477 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ca8f2-1ec5-4abe-b7d0-ab96f66b74ee None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:27.294573 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ca8f2-1ec5-4abe-b7d0-ab96f66b74ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:27.355136 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-580ca8f2-1ec5-4abe-b7d0-ab96f66b74ee None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:04:27.356420 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-580ca8f2-1ec5-4abe-b7d0-ab96f66b74ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:04:27.439514 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:27.454486 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 758/1508] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:04:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:04:27.465208 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76bb542-fe7d-41c2-aa41-fba3ab0f9d29 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:27.466530 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-f76bb542-fe7d-41c2-aa41-fba3ab0f9d29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:04:27.582783 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76bb542-fe7d-41c2-aa41-fba3ab0f9d29 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:27.656761 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:27.678706 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc7e46b-2236-476d-bdcb-45d1339693e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:27.679352 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7e46b-2236-476d-bdcb-45d1339693e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:27.679633 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7e46b-2236-476d-bdcb-45d1339693e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:27.679906 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7e46b-2236-476d-bdcb-45d1339693e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:27.681633 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-cdc7e46b-2236-476d-bdcb-45d1339693e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:04:27.798960 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc7e46b-2236-476d-bdcb-45d1339693e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:27.800367 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc7e46b-2236-476d-bdcb-45d1339693e1 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:27.821447 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 751/1509] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:04:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:04:28.221119 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ca8f2-1ec5-4abe-b7d0-ab96f66b74ee None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:28.222399 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-580ca8f2-1ec5-4abe-b7d0-ab96f66b74ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:04:28.239338 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ca8f2-1ec5-4abe-b7d0-ab96f66b74ee None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:28.353696 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390edd1-dc31-4b7a-b20f-523ea748c29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:28.353696 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390edd1-dc31-4b7a-b20f-523ea748c29a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:28.354874 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390edd1-dc31-4b7a-b20f-523ea748c29a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:28.354874 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390edd1-dc31-4b7a-b20f-523ea748c29a admin admin] PATH_INFO: `/v3/projects/1bd0f694d4ea4dc4bc923e08c2e2b503` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:28.365912 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5390edd1-dc31-4b7a-b20f-523ea748c29a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bd0f694d4ea4dc4bc923e08c2e2b503)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:28.367128 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: WARNING py.warnings [None req-5390edd1-dc31-4b7a-b20f-523ea748c29a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:04:28.367128 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: warnings.warn(msg) Aug 01 14:04:28.367128 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: Aug 01 14:04:28.368504 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5390edd1-dc31-4b7a-b20f-523ea748c29a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:28.438532 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5390edd1-dc31-4b7a-b20f-523ea748c29a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bd0f694d4ea4dc4bc923e08c2e2b503', 'request_id': 'req-5390edd1-dc31-4b7a-b20f-523ea748c29a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:04:28.445889 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5390edd1-dc31-4b7a-b20f-523ea748c29a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bd0f694d4ea4dc4bc923e08c2e2b503', 'request_id': 'req-5390edd1-dc31-4b7a-b20f-523ea748c29a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:04:28.456791 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5390edd1-dc31-4b7a-b20f-523ea748c29a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bd0f694d4ea4dc4bc923e08c2e2b503', 'request_id': 'req-5390edd1-dc31-4b7a-b20f-523ea748c29a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:04:28.498233 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5390edd1-dc31-4b7a-b20f-523ea748c29a admin admin] The token cache is being invalidate because project 1bd0f694d4ea4dc4bc923e08c2e2b503 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:04:28.498710 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5390edd1-dc31-4b7a-b20f-523ea748c29a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:04:28.499537 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5390edd1-dc31-4b7a-b20f-523ea748c29a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:04:28.502216 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 759/1510] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:04:28 2022] DELETE /identity/v3/projects/1bd0f694d4ea4dc4bc923e08c2e2b503 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:04:37.818972 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=62 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:38.494625 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:14.022032 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e262cc-4510-435e-833c-7d3afb0bf92e None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:14.022915 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e262cc-4510-435e-833c-7d3afb0bf92e None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:14.022915 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e262cc-4510-435e-833c-7d3afb0bf92e None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:14.025717 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 752/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:06:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:14.033930 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899e17b-43d5-4c9b-86da-31e490e6fa46 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:14.034529 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899e17b-43d5-4c9b-86da-31e490e6fa46 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:14.034529 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899e17b-43d5-4c9b-86da-31e490e6fa46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:14.129527 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b899e17b-43d5-4c9b-86da-31e490e6fa46 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:14.130667 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b899e17b-43d5-4c9b-86da-31e490e6fa46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:14.259762 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 760/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:06:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:06:14.419582 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e262cc-4510-435e-833c-7d3afb0bf92e None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:14.422208 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-19e262cc-4510-435e-833c-7d3afb0bf92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:14.570340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e262cc-4510-435e-833c-7d3afb0bf92e None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:14.577377 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd987a2f-336f-4d01-b2f7-2bfa1627026d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:14.578010 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd987a2f-336f-4d01-b2f7-2bfa1627026d service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:14.578292 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd987a2f-336f-4d01-b2f7-2bfa1627026d service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:14.578575 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd987a2f-336f-4d01-b2f7-2bfa1627026d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:14.587478 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd987a2f-336f-4d01-b2f7-2bfa1627026d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:14.589140 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd987a2f-336f-4d01-b2f7-2bfa1627026d service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:14.600140 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 753/1513] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:06:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:06:15.495081 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23282ec-86f4-4a7e-88c3-f05747a9f047 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:15.495684 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23282ec-86f4-4a7e-88c3-f05747a9f047 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:15.495684 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23282ec-86f4-4a7e-88c3-f05747a9f047 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:15.497098 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 761/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:06:15.504165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ef5d1-a5a7-4d61-8c1f-482c62644fa2 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:15.504836 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ef5d1-a5a7-4d61-8c1f-482c62644fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:15.504836 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ef5d1-a5a7-4d61-8c1f-482c62644fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:15.536261 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-5e8ef5d1-a5a7-4d61-8c1f-482c62644fa2 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:15.537955 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5e8ef5d1-a5a7-4d61-8c1f-482c62644fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:15.546450 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 754/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:06:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:06:15.704607 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23282ec-86f4-4a7e-88c3-f05747a9f047 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:15.720824 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23282ec-86f4-4a7e-88c3-f05747a9f047 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:15.729239 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a7be2-d670-400f-a58a-e477c06ce97a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:15.729907 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a7be2-d670-400f-a58a-e477c06ce97a service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:15.730186 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a7be2-d670-400f-a58a-e477c06ce97a service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:15.730486 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a7be2-d670-400f-a58a-e477c06ce97a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:15.742021 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a7be2-d670-400f-a58a-e477c06ce97a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:15.744423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a7be2-d670-400f-a58a-e477c06ce97a service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:15.756638 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 762/1516] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:06:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:06:16.257788 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0dae66-fbcf-4ef2-8766-ad7e9bb32ff5 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:16.258117 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0dae66-fbcf-4ef2-8766-ad7e9bb32ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:16.258488 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0dae66-fbcf-4ef2-8766-ad7e9bb32ff5 None None] PATH_INFO: `/v3` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:16.260606 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 755/1517] 10.222.0.57 () {54 vars in 925 bytes} [Mon Aug 1 14:06:16 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:16.268894 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e00ff-682e-43b4-9299-84894431bd1d None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:16.269209 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e00ff-682e-43b4-9299-84894431bd1d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:16.269495 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e00ff-682e-43b4-9299-84894431bd1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:16.347237 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-129e00ff-682e-43b4-9299-84894431bd1d None None] MFA Rules not processed for user `58380e30789041e4ad9e8652bc628658`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:16.348614 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-129e00ff-682e-43b4-9299-84894431bd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:16.369408 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:16.430520 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 763/1518] 10.222.0.57 () {58 vars in 1015 bytes} [Mon Aug 1 14:06:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:06:20.327900 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34bc127-b5be-4359-b181-7b05a84fff44 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:20.328730 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34bc127-b5be-4359-b181-7b05a84fff44 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:20.328730 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34bc127-b5be-4359-b181-7b05a84fff44 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:20.329959 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 756/1519] 10.222.0.45 () {54 vars in 961 bytes} [Mon Aug 1 14:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:20.338720 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd7c55-3f44-4b40-b50e-2c5a56c820e1 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:20.339244 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd7c55-3f44-4b40-b50e-2c5a56c820e1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:20.339783 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd7c55-3f44-4b40-b50e-2c5a56c820e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:20.371469 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c7bd7c55-3f44-4b40-b50e-2c5a56c820e1 None None] MFA Rules not processed for user `58380e30789041e4ad9e8652bc628658`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:20.373373 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c7bd7c55-3f44-4b40-b50e-2c5a56c820e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:20.384879 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 764/1520] 10.222.0.45 () {58 vars in 1058 bytes} [Mon Aug 1 14:06:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:06:20.393101 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34bc127-b5be-4359-b181-7b05a84fff44 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:20.410534 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34bc127-b5be-4359-b181-7b05a84fff44 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:20.419078 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fbf116-90d6-4eb2-9a3d-e89bf39c0f88 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58380e30789041e4ad9e8652bc628658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:20.420088 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbf116-90d6-4eb2-9a3d-e89bf39c0f88 service placement] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:20.420509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbf116-90d6-4eb2-9a3d-e89bf39c0f88 service placement] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:20.420988 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbf116-90d6-4eb2-9a3d-e89bf39c0f88 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:20.435902 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fbf116-90d6-4eb2-9a3d-e89bf39c0f88 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:20.438778 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fbf116-90d6-4eb2-9a3d-e89bf39c0f88 service placement] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:20.455739 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 757/1521] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:06:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:06:27.026734 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f206d-b3d4-4633-ab40-edd17016d073 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:27.028134 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f206d-b3d4-4633-ab40-edd17016d073 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:27.028470 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f206d-b3d4-4633-ab40-edd17016d073 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:27.029857 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 765/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:06:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:27.035041 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e20ea-f0eb-4132-90ae-b8b2abb13ae6 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:27.035489 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e20ea-f0eb-4132-90ae-b8b2abb13ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:27.035598 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e20ea-f0eb-4132-90ae-b8b2abb13ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:27.057334 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-d17e20ea-f0eb-4132-90ae-b8b2abb13ae6 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:27.058546 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d17e20ea-f0eb-4132-90ae-b8b2abb13ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:27.067468 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 758/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:06:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:06:27.224020 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f206d-b3d4-4633-ab40-edd17016d073 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:27.243788 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f206d-b3d4-4633-ab40-edd17016d073 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:27.251149 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc825c-402f-4e45-bc0b-c050e2361599 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:27.251778 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc825c-402f-4e45-bc0b-c050e2361599 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:27.252056 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc825c-402f-4e45-bc0b-c050e2361599 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:27.252443 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc825c-402f-4e45-bc0b-c050e2361599 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:27.263596 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc825c-402f-4e45-bc0b-c050e2361599 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:27.266418 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc825c-402f-4e45-bc0b-c050e2361599 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:27.276847 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 766/1524] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:06:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:06:28.091245 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a359c2-0829-46f2-ae0f-78f6dc4df617 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:28.091897 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a359c2-0829-46f2-ae0f-78f6dc4df617 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:28.091897 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a359c2-0829-46f2-ae0f-78f6dc4df617 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:28.093206 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 759/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:06:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:28.099211 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a135877-0fb9-45ee-8d6a-e7830e0a6cf3 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:28.099761 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a135877-0fb9-45ee-8d6a-e7830e0a6cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:28.099761 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a135877-0fb9-45ee-8d6a-e7830e0a6cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:28.125482 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-6a135877-0fb9-45ee-8d6a-e7830e0a6cf3 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:28.126726 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6a135877-0fb9-45ee-8d6a-e7830e0a6cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:28.135721 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 767/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:06:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:06:28.295189 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a359c2-0829-46f2-ae0f-78f6dc4df617 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:28.314893 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a359c2-0829-46f2-ae0f-78f6dc4df617 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:28.320457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4e09a-096f-4ad6-a4c0-41bd883fc065 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:28.321083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4e09a-096f-4ad6-a4c0-41bd883fc065 service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:28.321356 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4e09a-096f-4ad6-a4c0-41bd883fc065 service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:28.321610 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4e09a-096f-4ad6-a4c0-41bd883fc065 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:28.331304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd4e09a-096f-4ad6-a4c0-41bd883fc065 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:28.332940 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd4e09a-096f-4ad6-a4c0-41bd883fc065 service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:28.343288 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 760/1527] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:06:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:06:32.822586 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700271b7-59ad-41ac-85fc-a730145bda5a None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:32.823467 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700271b7-59ad-41ac-85fc-a730145bda5a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:32.823467 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700271b7-59ad-41ac-85fc-a730145bda5a None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:32.825227 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 768/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:06:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:32.833647 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072207c3-27af-4a23-a360-7ec8d6f228cf None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:32.834477 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072207c3-27af-4a23-a360-7ec8d6f228cf None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:32.834477 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072207c3-27af-4a23-a360-7ec8d6f228cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:32.868893 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-072207c3-27af-4a23-a360-7ec8d6f228cf None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:32.870126 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-072207c3-27af-4a23-a360-7ec8d6f228cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:32.879960 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 761/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:06:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:06:32.962047 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700271b7-59ad-41ac-85fc-a730145bda5a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:32.964655 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-700271b7-59ad-41ac-85fc-a730145bda5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:33.079168 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700271b7-59ad-41ac-85fc-a730145bda5a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:33.085441 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437545d4-c2fb-4193-aa2d-19478ee8536c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:33.086526 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437545d4-c2fb-4193-aa2d-19478ee8536c service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:33.087010 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437545d4-c2fb-4193-aa2d-19478ee8536c service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:33.087518 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437545d4-c2fb-4193-aa2d-19478ee8536c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:33.102663 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437545d4-c2fb-4193-aa2d-19478ee8536c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:33.104148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437545d4-c2fb-4193-aa2d-19478ee8536c service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:33.115181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 769/1530] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:06:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:06:35.129739 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874eadc0-d558-489b-a83c-b2d1bcf7330c None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:35.130326 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874eadc0-d558-489b-a83c-b2d1bcf7330c None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:35.130326 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874eadc0-d558-489b-a83c-b2d1bcf7330c None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:35.131674 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 762/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:35.137508 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b00d6-62a8-4bc3-8523-d2b76b24aecc None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:35.138040 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b00d6-62a8-4bc3-8523-d2b76b24aecc None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:35.138040 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b00d6-62a8-4bc3-8523-d2b76b24aecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:35.161962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b32b00d6-62a8-4bc3-8523-d2b76b24aecc None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:35.163500 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b32b00d6-62a8-4bc3-8523-d2b76b24aecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:35.173873 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 770/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:06:35.210094 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874eadc0-d558-489b-a83c-b2d1bcf7330c None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:35.222383 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874eadc0-d558-489b-a83c-b2d1bcf7330c None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:35.226412 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67640e-28fc-4e63-a7b1-3200f2c2a26d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:35.227036 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67640e-28fc-4e63-a7b1-3200f2c2a26d service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:35.227314 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67640e-28fc-4e63-a7b1-3200f2c2a26d service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:35.227586 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67640e-28fc-4e63-a7b1-3200f2c2a26d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:35.237639 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a67640e-28fc-4e63-a7b1-3200f2c2a26d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:35.239089 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a67640e-28fc-4e63-a7b1-3200f2c2a26d service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:35.248780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 763/1533] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:06:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:06:38.746062 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba1e7f-e038-4dba-b5bd-36aa6d20768a None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:38.746710 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba1e7f-e038-4dba-b5bd-36aa6d20768a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:38.746710 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba1e7f-e038-4dba-b5bd-36aa6d20768a None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:38.748030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 771/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:06:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:38.756699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a392e-a51d-4601-bcfa-baf874735de1 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:38.757155 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a392e-a51d-4601-bcfa-baf874735de1 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:38.757268 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a392e-a51d-4601-bcfa-baf874735de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:38.769059 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=81 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:38.782950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-215a392e-a51d-4601-bcfa-baf874735de1 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:38.784167 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-215a392e-a51d-4601-bcfa-baf874735de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:38.792640 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 764/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:06:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:06:39.100366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba1e7f-e038-4dba-b5bd-36aa6d20768a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:39.103487 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-64ba1e7f-e038-4dba-b5bd-36aa6d20768a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:39.183199 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:39.200892 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba1e7f-e038-4dba-b5bd-36aa6d20768a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:39.205149 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4790c38-d32f-4f83-89ad-399a8206220f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:39.205784 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4790c38-d32f-4f83-89ad-399a8206220f service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:39.206059 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4790c38-d32f-4f83-89ad-399a8206220f service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:39.206325 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4790c38-d32f-4f83-89ad-399a8206220f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:39.217240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4790c38-d32f-4f83-89ad-399a8206220f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:39.218711 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4790c38-d32f-4f83-89ad-399a8206220f service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:39.229257 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 772/1536] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:06:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:06:41.303805 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287b8f7-dab6-499c-ad07-e4b500cf72e4 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:41.304436 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287b8f7-dab6-499c-ad07-e4b500cf72e4 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:41.304436 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287b8f7-dab6-499c-ad07-e4b500cf72e4 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:41.305753 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 765/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:06:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:41.311919 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c71a9-2e94-40be-a099-c2ca4b3bc7da None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:41.312466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c71a9-2e94-40be-a099-c2ca4b3bc7da None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:41.312466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c71a9-2e94-40be-a099-c2ca4b3bc7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:41.339077 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-373c71a9-2e94-40be-a099-c2ca4b3bc7da None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:41.341671 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-373c71a9-2e94-40be-a099-c2ca4b3bc7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:41.357410 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 773/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:06:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:06:41.670602 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287b8f7-dab6-499c-ad07-e4b500cf72e4 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:41.673724 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d287b8f7-dab6-499c-ad07-e4b500cf72e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:41.693350 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287b8f7-dab6-499c-ad07-e4b500cf72e4 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:41.699807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9458b-4da4-4428-9aff-d0461e60b5d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:41.700639 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9458b-4da4-4428-9aff-d0461e60b5d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:41.700968 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9458b-4da4-4428-9aff-d0461e60b5d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:41.701315 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9458b-4da4-4428-9aff-d0461e60b5d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:41.716028 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b9458b-4da4-4428-9aff-d0461e60b5d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:41.717505 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b9458b-4da4-4428-9aff-d0461e60b5d7 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:41.727356 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 766/1539] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:06:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:06:51.329668 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=11 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:51.722901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=8 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:12.478943 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e9125-07c2-4019-af45-e980e810ca04 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:12.479710 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e9125-07c2-4019-af45-e980e810ca04 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:12.479710 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e9125-07c2-4019-af45-e980e810ca04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:12.508172 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-927e9125-07c2-4019-af45-e980e810ca04 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:12.509566 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-927e9125-07c2-4019-af45-e980e810ca04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:12.520134 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 774/1540] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:12.530534 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3e8b3-3f77-431f-97fc-9c7ae1d2ca67 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:12.531172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3e8b3-3f77-431f-97fc-9c7ae1d2ca67 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:12.531172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3e8b3-3f77-431f-97fc-9c7ae1d2ca67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:12.573637 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-faf3e8b3-3f77-431f-97fc-9c7ae1d2ca67 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:12.574923 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-faf3e8b3-3f77-431f-97fc-9c7ae1d2ca67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:12.584130 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 767/1541] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:12.589011 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e9125-07c2-4019-af45-e980e810ca04 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:12.607444 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e9125-07c2-4019-af45-e980e810ca04 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:12.616638 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e481bce-7bcd-4d78-82ca-ca6228ed39dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:12.617708 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e481bce-7bcd-4d78-82ca-ca6228ed39dd admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:12.618201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e481bce-7bcd-4d78-82ca-ca6228ed39dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:12.618704 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e481bce-7bcd-4d78-82ca-ca6228ed39dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:12.619647 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e481bce-7bcd-4d78-82ca-ca6228ed39dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:12.620190 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e481bce-7bcd-4d78-82ca-ca6228ed39dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:12.622614 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e481bce-7bcd-4d78-82ca-ca6228ed39dd admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:12.644825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 775/1542] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:08:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:12.653721 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf3e8b3-3f77-431f-97fc-9c7ae1d2ca67 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:12.664072 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf3e8b3-3f77-431f-97fc-9c7ae1d2ca67 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:12.672986 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b017a0-68f4-46cb-a6f6-cb30069fc999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:12.673653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b017a0-68f4-46cb-a6f6-cb30069fc999 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:12.674381 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b017a0-68f4-46cb-a6f6-cb30069fc999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:12.674964 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b017a0-68f4-46cb-a6f6-cb30069fc999 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:12.675783 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b017a0-68f4-46cb-a6f6-cb30069fc999 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:12.678374 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b017a0-68f4-46cb-a6f6-cb30069fc999 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:12.744971 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 768/1543] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:12 2022] POST /identity/v3/projects => generated 377 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:12.752937 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e481bce-7bcd-4d78-82ca-ca6228ed39dd admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:12.771930 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e481bce-7bcd-4d78-82ca-ca6228ed39dd admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:12.854758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8a2a8-25d2-4580-ae87-d2fd2ffab9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:12.856045 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8a2a8-25d2-4580-ae87-d2fd2ffab9fd admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:12.856577 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8a2a8-25d2-4580-ae87-d2fd2ffab9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:12.857073 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8a2a8-25d2-4580-ae87-d2fd2ffab9fd admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:12.858040 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a8a2a8-25d2-4580-ae87-d2fd2ffab9fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:12.860260 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a8a2a8-25d2-4580-ae87-d2fd2ffab9fd admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:12.890956 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 776/1544] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:12 2022] POST /identity/v3/users => generated 289 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:12.895664 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b017a0-68f4-46cb-a6f6-cb30069fc999 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:12.913370 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b017a0-68f4-46cb-a6f6-cb30069fc999 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:12.919143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb802d07-b4d6-4db3-b9aa-b0db719fc847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:12.919818 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb802d07-b4d6-4db3-b9aa-b0db719fc847 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:12.920098 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb802d07-b4d6-4db3-b9aa-b0db719fc847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:12.920582 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb802d07-b4d6-4db3-b9aa-b0db719fc847 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:12.921570 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb802d07-b4d6-4db3-b9aa-b0db719fc847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:12.923823 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb802d07-b4d6-4db3-b9aa-b0db719fc847 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:12.936818 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 769/1545] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:12.941913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8a2a8-25d2-4580-ae87-d2fd2ffab9fd admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:12.960473 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8a2a8-25d2-4580-ae87-d2fd2ffab9fd admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:12.969829 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eee4ef-92f4-4ffe-9b0c-6e17341fe7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:12.970892 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eee4ef-92f4-4ffe-9b0c-6e17341fe7ff admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:12.971464 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eee4ef-92f4-4ffe-9b0c-6e17341fe7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:12.972121 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eee4ef-92f4-4ffe-9b0c-6e17341fe7ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:12.973314 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eee4ef-92f4-4ffe-9b0c-6e17341fe7ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:12.975667 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eee4ef-92f4-4ffe-9b0c-6e17341fe7ff admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:12.993166 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 777/1546] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:12 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:12.999467 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb802d07-b4d6-4db3-b9aa-b0db719fc847 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:13.010643 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb802d07-b4d6-4db3-b9aa-b0db719fc847 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:13.024168 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e110582-cb9d-4860-b1af-60c10282a265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:13.024985 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e110582-cb9d-4860-b1af-60c10282a265 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:13.025263 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e110582-cb9d-4860-b1af-60c10282a265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:13.025537 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e110582-cb9d-4860-b1af-60c10282a265 admin admin] PATH_INFO: `/v3/projects/1081ae5fd3e9443c80626517ae9c7d77/users/87f35e960544403ea30ba36d878a86cb/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:13.056385 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e110582-cb9d-4860-b1af-60c10282a265 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1081ae5fd3e9443c80626517ae9c7d77, user_id=87f35e960544403ea30ba36d878a86cb, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:13.059098 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e110582-cb9d-4860-b1af-60c10282a265 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:13.079759 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 770/1547] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:12 2022] PUT /identity/v3/projects/1081ae5fd3e9443c80626517ae9c7d77/users/87f35e960544403ea30ba36d878a86cb/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:08:13.086784 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef8f04-0483-40f2-91d9-6eeda98da85e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:13.087977 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef8f04-0483-40f2-91d9-6eeda98da85e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:13.088983 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef8f04-0483-40f2-91d9-6eeda98da85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:13.099267 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5fe0e3-1d92-4403-a135-a326be2867a6 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:13.099628 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5fe0e3-1d92-4403-a135-a326be2867a6 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:13.099872 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5fe0e3-1d92-4403-a135-a326be2867a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:13.132504 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-efef8f04-0483-40f2-91d9-6eeda98da85e None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:13.135354 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-efef8f04-0483-40f2-91d9-6eeda98da85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:13.144133 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-4d5fe0e3-1d92-4403-a135-a326be2867a6 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:13.146129 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-4d5fe0e3-1d92-4403-a135-a326be2867a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:13.290653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 771/1548] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:08:13.291995 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3617928-2851-43a5-89b2-05ef1c0f9cb1 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:13.295290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3617928-2851-43a5-89b2-05ef1c0f9cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:13.295664 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3617928-2851-43a5-89b2-05ef1c0f9cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:13.299018 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 778/1549] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:13.301559 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08155c3e-28ae-409c-940a-5ffabdcd7a8c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:13.302396 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08155c3e-28ae-409c-940a-5ffabdcd7a8c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:13.306040 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08155c3e-28ae-409c-940a-5ffabdcd7a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:13.336493 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-a3617928-2851-43a5-89b2-05ef1c0f9cb1 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:13.338074 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a3617928-2851-43a5-89b2-05ef1c0f9cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:13.343028 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-08155c3e-28ae-409c-940a-5ffabdcd7a8c None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:13.344500 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-08155c3e-28ae-409c-940a-5ffabdcd7a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:13.349470 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 772/1550] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:13.351009 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6baff-a16d-4012-9ddb-34c03e84b24b None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:13.351635 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6baff-a16d-4012-9ddb-34c03e84b24b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:13.352101 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6baff-a16d-4012-9ddb-34c03e84b24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:13.354028 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 779/1551] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:13.355166 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08155c3e-28ae-409c-940a-5ffabdcd7a8c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:13.384627 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-14b6baff-a16d-4012-9ddb-34c03e84b24b None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:13.386085 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-14b6baff-a16d-4012-9ddb-34c03e84b24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:13.395265 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 773/1552] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:13.400860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3205b73-8ce5-410f-9928-4f5ad026a6bc None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:13.400860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3205b73-8ce5-410f-9928-4f5ad026a6bc None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:13.400860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3205b73-8ce5-410f-9928-4f5ad026a6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:13.425149 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-d3205b73-8ce5-410f-9928-4f5ad026a6bc None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:13.426603 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d3205b73-8ce5-410f-9928-4f5ad026a6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:13.434709 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08155c3e-28ae-409c-940a-5ffabdcd7a8c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:13.435696 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 774/1553] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:13.437106 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440efd7d-d7a8-476c-a32f-620ebe80f558 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:13.437404 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440efd7d-d7a8-476c-a32f-620ebe80f558 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:13.437688 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440efd7d-d7a8-476c-a32f-620ebe80f558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:13.439926 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86eb766-3ce5-4a47-968a-fa15a04af5a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:13.440877 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86eb766-3ce5-4a47-968a-fa15a04af5a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:13.441309 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86eb766-3ce5-4a47-968a-fa15a04af5a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:13.441756 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86eb766-3ce5-4a47-968a-fa15a04af5a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:13.449862 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:13.453037 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86eb766-3ce5-4a47-968a-fa15a04af5a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:13.454902 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86eb766-3ce5-4a47-968a-fa15a04af5a4 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:13.458510 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-440efd7d-d7a8-476c-a32f-620ebe80f558 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:13.459736 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-440efd7d-d7a8-476c-a32f-620ebe80f558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:13.467870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 780/1554] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:13.468981 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919d86a-6243-4250-bfe9-b2c3c2675ee5 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:13.469281 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919d86a-6243-4250-bfe9-b2c3c2675ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:13.469608 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919d86a-6243-4250-bfe9-b2c3c2675ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:13.472574 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 775/1555] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:13.473879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998114a9-0591-4b9b-ae9d-682d88a75218 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:13.474168 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998114a9-0591-4b9b-ae9d-682d88a75218 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:13.474431 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998114a9-0591-4b9b-ae9d-682d88a75218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:13.497593 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-998114a9-0591-4b9b-ae9d-682d88a75218 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:13.499271 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-998114a9-0591-4b9b-ae9d-682d88a75218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:13.499778 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8919d86a-6243-4250-bfe9-b2c3c2675ee5 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:13.501273 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8919d86a-6243-4250-bfe9-b2c3c2675ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:13.508975 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 776/1556] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:13.510660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091bf725-10cc-4d2e-b3d5-ddc81e7aec30 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:13.511149 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091bf725-10cc-4d2e-b3d5-ddc81e7aec30 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:13.511421 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091bf725-10cc-4d2e-b3d5-ddc81e7aec30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:13.514805 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 781/1557] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:13.517613 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8919d86a-6243-4250-bfe9-b2c3c2675ee5 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:13.531568 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8919d86a-6243-4250-bfe9-b2c3c2675ee5 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:13.537200 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-091bf725-10cc-4d2e-b3d5-ddc81e7aec30 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:13.538450 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-091bf725-10cc-4d2e-b3d5-ddc81e7aec30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:13.545563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb3ae7d-3a03-41bc-9df6-8b13fe0a31cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:13.546801 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3ae7d-3a03-41bc-9df6-8b13fe0a31cf admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:13.547388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3ae7d-3a03-41bc-9df6-8b13fe0a31cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:13.547676 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3ae7d-3a03-41bc-9df6-8b13fe0a31cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:13.557418 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb3ae7d-3a03-41bc-9df6-8b13fe0a31cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:13.557709 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 777/1558] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:13.557855 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb3ae7d-3a03-41bc-9df6-8b13fe0a31cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:13.558157 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865d2cc-a3b4-45c1-b864-9f4341be55a4 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:13.558157 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865d2cc-a3b4-45c1-b864-9f4341be55a4 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:13.558157 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865d2cc-a3b4-45c1-b864-9f4341be55a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:13.558515 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb3ae7d-3a03-41bc-9df6-8b13fe0a31cf admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:13.568421 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 782/1559] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:13.568421 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40398e-4a37-4cca-a321-49cae8c62e2c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:13.568421 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40398e-4a37-4cca-a321-49cae8c62e2c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:13.568421 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40398e-4a37-4cca-a321-49cae8c62e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:13.585858 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-9865d2cc-a3b4-45c1-b864-9f4341be55a4 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:13.587473 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-9865d2cc-a3b4-45c1-b864-9f4341be55a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:13.590399 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-de40398e-4a37-4cca-a321-49cae8c62e2c None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:13.591837 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-de40398e-4a37-4cca-a321-49cae8c62e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:13.597688 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 778/1560] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:13.600674 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84b66e-a6e2-4fd9-821c-d16cf966af36 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:13.600674 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84b66e-a6e2-4fd9-821c-d16cf966af36 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:13.600674 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84b66e-a6e2-4fd9-821c-d16cf966af36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:13.611191 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 783/1561] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:13.611926 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40398e-4a37-4cca-a321-49cae8c62e2c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:13.623118 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-9b84b66e-a6e2-4fd9-821c-d16cf966af36 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:13.623840 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40398e-4a37-4cca-a321-49cae8c62e2c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:13.624676 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-9b84b66e-a6e2-4fd9-821c-d16cf966af36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:13.629724 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b35b8-e576-4ed4-8c70-ee9a0ec93835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:13.630136 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b35b8-e576-4ed4-8c70-ee9a0ec93835 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:13.630410 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b35b8-e576-4ed4-8c70-ee9a0ec93835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:13.630739 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b35b8-e576-4ed4-8c70-ee9a0ec93835 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:13.631511 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1b35b8-e576-4ed4-8c70-ee9a0ec93835 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:13.631819 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1b35b8-e576-4ed4-8c70-ee9a0ec93835 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:13.633519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1b35b8-e576-4ed4-8c70-ee9a0ec93835 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:13.636314 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 779/1562] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:13.636314 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b84b66e-a6e2-4fd9-821c-d16cf966af36 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:13.649822 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 784/1563] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:13.657816 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b84b66e-a6e2-4fd9-821c-d16cf966af36 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:13.663606 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3bbff-c93f-48e0-9430-9273cd82cf11 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:13.663950 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3bbff-c93f-48e0-9430-9273cd82cf11 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:13.664265 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3bbff-c93f-48e0-9430-9273cd82cf11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:13.666162 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d4487-687d-42e5-9a6c-df84f347e2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:13.666811 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d4487-687d-42e5-9a6c-df84f347e2d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:13.667088 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d4487-687d-42e5-9a6c-df84f347e2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:13.667427 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d4487-687d-42e5-9a6c-df84f347e2d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:13.668076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d4487-687d-42e5-9a6c-df84f347e2d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:13.668447 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d4487-687d-42e5-9a6c-df84f347e2d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:13.669586 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: WARNING py.warnings [None req-075d4487-687d-42e5-9a6c-df84f347e2d3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:08:13.669586 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: warnings.warn(msg) Aug 01 14:08:13.669586 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: Aug 01 14:08:13.670565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d4487-687d-42e5-9a6c-df84f347e2d3 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:13.686525 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 780/1564] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:13.687244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d4487-687d-42e5-9a6c-df84f347e2d3 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:13.692008 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-92d3bbff-c93f-48e0-9430-9273cd82cf11 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:13.695334 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-92d3bbff-c93f-48e0-9430-9273cd82cf11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:13.699434 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d4487-687d-42e5-9a6c-df84f347e2d3 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:13.704743 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f5cc2-c745-4ab9-9106-6e9e9cb55051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:13.705502 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f5cc2-c745-4ab9-9106-6e9e9cb55051 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:13.705833 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 785/1565] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:13.709988 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f5cc2-c745-4ab9-9106-6e9e9cb55051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:13.709988 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f5cc2-c745-4ab9-9106-6e9e9cb55051 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:13.709988 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f5cc2-c745-4ab9-9106-6e9e9cb55051 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:13.710236 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf78df-cfb4-418d-ae67-820eb3378115 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:13.710236 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf78df-cfb4-418d-ae67-820eb3378115 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:13.710236 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf78df-cfb4-418d-ae67-820eb3378115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:13.710462 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f5cc2-c745-4ab9-9106-6e9e9cb55051 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:13.740401 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-5baf78df-cfb4-418d-ae67-820eb3378115 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:13.742145 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5baf78df-cfb4-418d-ae67-820eb3378115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:13.745265 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:13.752398 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 786/1566] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:13.754563 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 781/1567] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:13 2022] POST /identity/v3/projects => generated 403 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:13.757445 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf78df-cfb4-418d-ae67-820eb3378115 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:13.758934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f5cc2-c745-4ab9-9106-6e9e9cb55051 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:13.771269 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf78df-cfb4-418d-ae67-820eb3378115 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:13.848819 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f5cc2-c745-4ab9-9106-6e9e9cb55051 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:13.853704 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aee29c-a696-472e-a604-6bee861f3148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:13.854244 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b9d565-4a60-4b76-ac40-15a1aca44817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:13.854500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aee29c-a696-472e-a604-6bee861f3148 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:13.854859 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aee29c-a696-472e-a604-6bee861f3148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:13.855199 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aee29c-a696-472e-a604-6bee861f3148 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:13.855373 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9d565-4a60-4b76-ac40-15a1aca44817 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:13.855991 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9d565-4a60-4b76-ac40-15a1aca44817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:13.856120 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aee29c-a696-472e-a604-6bee861f3148 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:13.856429 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9d565-4a60-4b76-ac40-15a1aca44817 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:13.856638 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aee29c-a696-472e-a604-6bee861f3148 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:13.857742 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b9d565-4a60-4b76-ac40-15a1aca44817 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:13.858422 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aee29c-a696-472e-a604-6bee861f3148 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:13.858562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b9d565-4a60-4b76-ac40-15a1aca44817 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:13.862130 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b9d565-4a60-4b76-ac40-15a1aca44817 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:13.884956 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 787/1568] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:13.886617 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b9d565-4a60-4b76-ac40-15a1aca44817 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:13.890218 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 782/1569] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:13.891662 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aee29c-a696-472e-a604-6bee861f3148 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:13.902527 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b9d565-4a60-4b76-ac40-15a1aca44817 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:13.909737 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5491c8-3336-4c9d-9579-d63fb4d56021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:13.910756 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5491c8-3336-4c9d-9579-d63fb4d56021 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:13.911336 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5491c8-3336-4c9d-9579-d63fb4d56021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:13.911760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5491c8-3336-4c9d-9579-d63fb4d56021 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:13.912188 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aee29c-a696-472e-a604-6bee861f3148 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:13.912924 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5491c8-3336-4c9d-9579-d63fb4d56021 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:13.914234 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: WARNING py.warnings [None req-2a5491c8-3336-4c9d-9579-d63fb4d56021 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:08:13.914234 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: warnings.warn(msg) Aug 01 14:08:13.914234 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: Aug 01 14:08:13.916326 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5491c8-3336-4c9d-9579-d63fb4d56021 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:13.921775 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfeec8-2af6-4ae7-8251-563e1b54b2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:13.923433 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfeec8-2af6-4ae7-8251-563e1b54b2db admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:13.925024 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfeec8-2af6-4ae7-8251-563e1b54b2db admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:13.925629 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfeec8-2af6-4ae7-8251-563e1b54b2db admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:13.927091 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cfeec8-2af6-4ae7-8251-563e1b54b2db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:13.931431 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cfeec8-2af6-4ae7-8251-563e1b54b2db admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:13.956018 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 788/1570] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:13 2022] POST /identity/v3/projects => generated 393 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:13.957394 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5491c8-3336-4c9d-9579-d63fb4d56021 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:14.001156 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 783/1571] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:13 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:14.002183 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfeec8-2af6-4ae7-8251-563e1b54b2db admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:14.090653 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5491c8-3336-4c9d-9579-d63fb4d56021 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:14.117791 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:14.136024 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfeec8-2af6-4ae7-8251-563e1b54b2db admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:14.141668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b032a-16c8-4c24-ad55-c3e65d1311b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:14.142321 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b032a-16c8-4c24-ad55-c3e65d1311b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:14.142598 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b032a-16c8-4c24-ad55-c3e65d1311b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:14.143332 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b032a-16c8-4c24-ad55-c3e65d1311b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:14.143608 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b032a-16c8-4c24-ad55-c3e65d1311b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:14.144686 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: WARNING py.warnings [None req-ee9b032a-16c8-4c24-ad55-c3e65d1311b0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:08:14.144686 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: warnings.warn(msg) Aug 01 14:08:14.144686 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: Aug 01 14:08:14.145917 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b032a-16c8-4c24-ad55-c3e65d1311b0 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:14.173140 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 784/1572] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:14 2022] POST /identity/v3/users => generated 302 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:14.177343 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b032a-16c8-4c24-ad55-c3e65d1311b0 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:14.183767 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02510490-9947-466e-b97a-0751ca4f34ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:14.185499 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02510490-9947-466e-b97a-0751ca4f34ac admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:14.186125 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02510490-9947-466e-b97a-0751ca4f34ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:14.186806 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02510490-9947-466e-b97a-0751ca4f34ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:14.188054 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02510490-9947-466e-b97a-0751ca4f34ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:14.188974 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02510490-9947-466e-b97a-0751ca4f34ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:14.189542 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b032a-16c8-4c24-ad55-c3e65d1311b0 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:14.190901 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02510490-9947-466e-b97a-0751ca4f34ac admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:14.195776 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3ad4f-885d-4229-beb2-9acdf76d48b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:14.196020 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3ad4f-885d-4229-beb2-9acdf76d48b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:14.196236 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3ad4f-885d-4229-beb2-9acdf76d48b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:14.196537 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3ad4f-885d-4229-beb2-9acdf76d48b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:14.197252 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c3ad4f-885d-4229-beb2-9acdf76d48b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:14.197590 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c3ad4f-885d-4229-beb2-9acdf76d48b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:14.199559 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c3ad4f-885d-4229-beb2-9acdf76d48b0 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:14.220301 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 789/1573] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:14.220301 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02510490-9947-466e-b97a-0751ca4f34ac admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:14.220650 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 785/1574] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:14.220650 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3ad4f-885d-4229-beb2-9acdf76d48b0 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:14.228477 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3ad4f-885d-4229-beb2-9acdf76d48b0 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:14.231214 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02510490-9947-466e-b97a-0751ca4f34ac admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:14.234045 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c4e081-da0b-4774-abdc-231c2fe30b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:14.234771 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4e081-da0b-4774-abdc-231c2fe30b27 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:14.235129 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4e081-da0b-4774-abdc-231c2fe30b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:14.235561 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4e081-da0b-4774-abdc-231c2fe30b27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:14.236362 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c4e081-da0b-4774-abdc-231c2fe30b27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:14.237038 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c126f6-30ed-4440-a886-22565c5d4916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:14.237760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c126f6-30ed-4440-a886-22565c5d4916 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:14.238117 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c126f6-30ed-4440-a886-22565c5d4916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:14.238457 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c126f6-30ed-4440-a886-22565c5d4916 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:14.238737 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c4e081-da0b-4774-abdc-231c2fe30b27 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:14.239269 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c126f6-30ed-4440-a886-22565c5d4916 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:14.242002 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c126f6-30ed-4440-a886-22565c5d4916 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:14.279882 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 786/1575] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:14 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:14.280856 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 790/1576] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:14 2022] POST /identity/v3/projects => generated 385 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:14.280998 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c4e081-da0b-4774-abdc-231c2fe30b27 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:14.283056 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c126f6-30ed-4440-a886-22565c5d4916 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:14.334116 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:14.374354 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c126f6-30ed-4440-a886-22565c5d4916 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:14.382527 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f87798-377a-49d6-bbba-54aea6b81ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:14.383318 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f87798-377a-49d6-bbba-54aea6b81ef9 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:14.383759 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f87798-377a-49d6-bbba-54aea6b81ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:14.384153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f87798-377a-49d6-bbba-54aea6b81ef9 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:14.384989 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f87798-377a-49d6-bbba-54aea6b81ef9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:14.387587 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f87798-377a-49d6-bbba-54aea6b81ef9 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:14.396424 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c4e081-da0b-4774-abdc-231c2fe30b27 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:14.406461 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2da02-86a7-4a91-be49-74e8c6c59e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:14.407687 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2da02-86a7-4a91-be49-74e8c6c59e91 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:14.408465 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2da02-86a7-4a91-be49-74e8c6c59e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:14.409190 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2da02-86a7-4a91-be49-74e8c6c59e91 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:14.410134 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b2da02-86a7-4a91-be49-74e8c6c59e91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:14.412458 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b2da02-86a7-4a91-be49-74e8c6c59e91 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:14.420537 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 791/1577] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:14 2022] POST /identity/v3/users => generated 290 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:14.421745 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f87798-377a-49d6-bbba-54aea6b81ef9 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:14.442357 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f87798-377a-49d6-bbba-54aea6b81ef9 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:14.453901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 787/1578] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:14 2022] POST /identity/v3/users => generated 297 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:14.454776 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2da02-86a7-4a91-be49-74e8c6c59e91 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:14.456865 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d3caf-bd75-4a99-abf4-29a0cef149d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:14.458594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d3caf-bd75-4a99-abf4-29a0cef149d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:14.459412 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d3caf-bd75-4a99-abf4-29a0cef149d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:14.464734 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d3caf-bd75-4a99-abf4-29a0cef149d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:14.466593 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d3caf-bd75-4a99-abf4-29a0cef149d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:14.472963 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2da02-86a7-4a91-be49-74e8c6c59e91 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:14.473240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d3caf-bd75-4a99-abf4-29a0cef149d4 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:14.480227 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e47086-4624-424e-9d7e-3e84b3bf68c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:14.482286 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e47086-4624-424e-9d7e-3e84b3bf68c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:14.482286 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e47086-4624-424e-9d7e-3e84b3bf68c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:14.482286 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e47086-4624-424e-9d7e-3e84b3bf68c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:14.482814 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e47086-4624-424e-9d7e-3e84b3bf68c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:14.484911 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e47086-4624-424e-9d7e-3e84b3bf68c0 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:14.499043 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 792/1579] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:14.504439 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d3caf-bd75-4a99-abf4-29a0cef149d4 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:14.522905 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d3caf-bd75-4a99-abf4-29a0cef149d4 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:14.535087 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827aa93-b2c0-48ea-868c-3eb84784a8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:14.536981 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827aa93-b2c0-48ea-868c-3eb84784a8ce admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:14.537696 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827aa93-b2c0-48ea-868c-3eb84784a8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:14.538228 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827aa93-b2c0-48ea-868c-3eb84784a8ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:14.539907 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e827aa93-b2c0-48ea-868c-3eb84784a8ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:14.543871 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e827aa93-b2c0-48ea-868c-3eb84784a8ce admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:14.555491 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 788/1580] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:14 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:14.555491 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e47086-4624-424e-9d7e-3e84b3bf68c0 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:14.604773 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 793/1581] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:14 2022] POST /identity/v3/projects => generated 371 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:14.610138 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827aa93-b2c0-48ea-868c-3eb84784a8ce admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:14.657954 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e47086-4624-424e-9d7e-3e84b3bf68c0 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:14.694130 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:14.715696 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827aa93-b2c0-48ea-868c-3eb84784a8ce admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:14.723322 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055dac13-0ca4-4269-87e2-cf85344f3285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:14.724378 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055dac13-0ca4-4269-87e2-cf85344f3285 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:14.724677 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055dac13-0ca4-4269-87e2-cf85344f3285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:14.724962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055dac13-0ca4-4269-87e2-cf85344f3285 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:14.725894 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055dac13-0ca4-4269-87e2-cf85344f3285 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:14.729640 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055dac13-0ca4-4269-87e2-cf85344f3285 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:14.750285 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafd572-37cf-4dae-ad87-fa83af798b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:14.751307 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafd572-37cf-4dae-ad87-fa83af798b28 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:14.751907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafd572-37cf-4dae-ad87-fa83af798b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:14.752557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafd572-37cf-4dae-ad87-fa83af798b28 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:14.754806 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dafd572-37cf-4dae-ad87-fa83af798b28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:14.757175 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dafd572-37cf-4dae-ad87-fa83af798b28 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:14.759729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 794/1582] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:14 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:14.760800 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055dac13-0ca4-4269-87e2-cf85344f3285 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:14.788929 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055dac13-0ca4-4269-87e2-cf85344f3285 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:14.799118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de67788-fd62-4f6c-8587-4b8e62d72b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:14.800300 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de67788-fd62-4f6c-8587-4b8e62d72b3d admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:14.800809 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de67788-fd62-4f6c-8587-4b8e62d72b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:14.801670 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de67788-fd62-4f6c-8587-4b8e62d72b3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:14.803526 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de67788-fd62-4f6c-8587-4b8e62d72b3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:14.808464 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de67788-fd62-4f6c-8587-4b8e62d72b3d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:14.810195 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 789/1583] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:14 2022] POST /identity/v3/users => generated 286 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:14.814560 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafd572-37cf-4dae-ad87-fa83af798b28 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:14.837314 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafd572-37cf-4dae-ad87-fa83af798b28 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:14.837971 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 795/1584] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:14.839745 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de67788-fd62-4f6c-8587-4b8e62d72b3d admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:14.850059 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16825f1-e8a0-4ad3-94bb-162c171b01e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:14.851271 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16825f1-e8a0-4ad3-94bb-162c171b01e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:14.852633 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16825f1-e8a0-4ad3-94bb-162c171b01e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:14.853158 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16825f1-e8a0-4ad3-94bb-162c171b01e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:14.854494 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16825f1-e8a0-4ad3-94bb-162c171b01e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:14.859056 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16825f1-e8a0-4ad3-94bb-162c171b01e9 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:14.860141 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de67788-fd62-4f6c-8587-4b8e62d72b3d admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:14.869920 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d447b9-f602-4073-8205-ffd58783353c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:14.871447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d447b9-f602-4073-8205-ffd58783353c admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:14.871894 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d447b9-f602-4073-8205-ffd58783353c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:14.872507 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d447b9-f602-4073-8205-ffd58783353c admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:14.873791 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d447b9-f602-4073-8205-ffd58783353c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:14.877283 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d447b9-f602-4073-8205-ffd58783353c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:14.888461 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 790/1585] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:14.889844 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16825f1-e8a0-4ad3-94bb-162c171b01e9 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:14.901293 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 796/1586] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:14.902758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d447b9-f602-4073-8205-ffd58783353c admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:14.914251 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16825f1-e8a0-4ad3-94bb-162c171b01e9 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:14.921548 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d447b9-f602-4073-8205-ffd58783353c admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:14.924464 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46c2e6-090e-4bd9-b03b-a29bcb881bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:14.925468 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46c2e6-090e-4bd9-b03b-a29bcb881bcc admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:14.926085 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46c2e6-090e-4bd9-b03b-a29bcb881bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:14.926671 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46c2e6-090e-4bd9-b03b-a29bcb881bcc admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:14.927894 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb46c2e6-090e-4bd9-b03b-a29bcb881bcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:14.929711 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f9ff0-a670-4727-8df9-2ca500ec0a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:14.930842 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f9ff0-a670-4727-8df9-2ca500ec0a80 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:14.931226 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f9ff0-a670-4727-8df9-2ca500ec0a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:14.931703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f9ff0-a670-4727-8df9-2ca500ec0a80 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:14.931980 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb46c2e6-090e-4bd9-b03b-a29bcb881bcc admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:14.932962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f9ff0-a670-4727-8df9-2ca500ec0a80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:14.936697 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f9ff0-a670-4727-8df9-2ca500ec0a80 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:14.970742 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 791/1587] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:14 2022] POST /identity/v3/users => generated 287 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:14.970742 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46c2e6-090e-4bd9-b03b-a29bcb881bcc admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:14.976889 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:14.977615 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 797/1588] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:14 2022] POST /identity/v3/users => generated 286 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:14.979414 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f9ff0-a670-4727-8df9-2ca500ec0a80 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:14.990018 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46c2e6-090e-4bd9-b03b-a29bcb881bcc admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:14.998138 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c296f0-955c-4905-97ab-2915a3b97ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:14.998886 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c296f0-955c-4905-97ab-2915a3b97ee5 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:14.999230 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c296f0-955c-4905-97ab-2915a3b97ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:14.999568 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c296f0-955c-4905-97ab-2915a3b97ee5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:15.000524 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c296f0-955c-4905-97ab-2915a3b97ee5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:15.001875 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f9ff0-a670-4727-8df9-2ca500ec0a80 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:15.003067 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c296f0-955c-4905-97ab-2915a3b97ee5 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:15.011796 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29229fa-68ca-49a4-9dbd-8cbfbd7f658c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:15.012648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29229fa-68ca-49a4-9dbd-8cbfbd7f658c admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:15.013018 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29229fa-68ca-49a4-9dbd-8cbfbd7f658c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:15.013699 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29229fa-68ca-49a4-9dbd-8cbfbd7f658c admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:15.014625 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29229fa-68ca-49a4-9dbd-8cbfbd7f658c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:15.016877 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29229fa-68ca-49a4-9dbd-8cbfbd7f658c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:15.020636 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 792/1589] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:15.020636 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c296f0-955c-4905-97ab-2915a3b97ee5 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:15.034470 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 798/1590] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:15.034470 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29229fa-68ca-49a4-9dbd-8cbfbd7f658c admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:15.038463 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c296f0-955c-4905-97ab-2915a3b97ee5 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:15.046101 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29229fa-68ca-49a4-9dbd-8cbfbd7f658c admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:15.048655 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7d669d-3acb-4d2a-88ee-7ddbb1238e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:15.049930 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d669d-3acb-4d2a-88ee-7ddbb1238e93 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:15.050478 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d669d-3acb-4d2a-88ee-7ddbb1238e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:15.051048 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d669d-3acb-4d2a-88ee-7ddbb1238e93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:15.052152 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7d669d-3acb-4d2a-88ee-7ddbb1238e93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:15.052744 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d98979-af05-49e9-acfe-0d057ae3fd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:15.053566 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d98979-af05-49e9-acfe-0d057ae3fd01 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:15.053978 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d98979-af05-49e9-acfe-0d057ae3fd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:15.054373 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d98979-af05-49e9-acfe-0d057ae3fd01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:15.055503 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7d669d-3acb-4d2a-88ee-7ddbb1238e93 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:15.055640 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d98979-af05-49e9-acfe-0d057ae3fd01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:15.057539 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d98979-af05-49e9-acfe-0d057ae3fd01 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:15.085487 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 793/1591] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:15.085487 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7d669d-3acb-4d2a-88ee-7ddbb1238e93 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:15.089202 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 799/1592] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:15.090439 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d98979-af05-49e9-acfe-0d057ae3fd01 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:15.099231 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7d669d-3acb-4d2a-88ee-7ddbb1238e93 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:15.104772 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d98979-af05-49e9-acfe-0d057ae3fd01 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:15.105577 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f607cf6-e5e8-4190-94f0-f1a67ccdd4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:15.107045 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f607cf6-e5e8-4190-94f0-f1a67ccdd4f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:15.107045 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f607cf6-e5e8-4190-94f0-f1a67ccdd4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:15.107045 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f607cf6-e5e8-4190-94f0-f1a67ccdd4f5 admin admin] PATH_INFO: `/v3/projects/77b5cb16a7fe46a68be4a6c47f234e36/users/391bb00d7b25489985cbd5983ac41a19/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:15.114276 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42232a16-1f8e-4eb2-bfc1-fb8c1caaffc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:15.115615 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42232a16-1f8e-4eb2-bfc1-fb8c1caaffc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:15.115950 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42232a16-1f8e-4eb2-bfc1-fb8c1caaffc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:15.116230 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42232a16-1f8e-4eb2-bfc1-fb8c1caaffc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:15.117137 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42232a16-1f8e-4eb2-bfc1-fb8c1caaffc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:15.120827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42232a16-1f8e-4eb2-bfc1-fb8c1caaffc0 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:15.126852 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f607cf6-e5e8-4190-94f0-f1a67ccdd4f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77b5cb16a7fe46a68be4a6c47f234e36, user_id=391bb00d7b25489985cbd5983ac41a19, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:15.131306 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f607cf6-e5e8-4190-94f0-f1a67ccdd4f5 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:15.138823 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 800/1593] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:15.140329 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42232a16-1f8e-4eb2-bfc1-fb8c1caaffc0 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:15.154674 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42232a16-1f8e-4eb2-bfc1-fb8c1caaffc0 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:15.161508 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 794/1594] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:15 2022] PUT /identity/v3/projects/77b5cb16a7fe46a68be4a6c47f234e36/users/391bb00d7b25489985cbd5983ac41a19/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:15.162513 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f607cf6-e5e8-4190-94f0-f1a67ccdd4f5 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:15.286834 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f607cf6-e5e8-4190-94f0-f1a67ccdd4f5 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:15.294116 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c1d6e1-7822-417f-8e34-f0d6e348dddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:15.295942 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1d6e1-7822-417f-8e34-f0d6e348dddc admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:15.296237 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1d6e1-7822-417f-8e34-f0d6e348dddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:15.296571 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1d6e1-7822-417f-8e34-f0d6e348dddc admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:15.297423 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c1d6e1-7822-417f-8e34-f0d6e348dddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:15.302328 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c1d6e1-7822-417f-8e34-f0d6e348dddc admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:15.321825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a4868-94b1-497d-a6a4-7ed1b54574eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:15.323128 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a4868-94b1-497d-a6a4-7ed1b54574eb admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:15.323664 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a4868-94b1-497d-a6a4-7ed1b54574eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:15.324149 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a4868-94b1-497d-a6a4-7ed1b54574eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:15.324326 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 795/1595] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:15.325289 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a4868-94b1-497d-a6a4-7ed1b54574eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:15.325656 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c1d6e1-7822-417f-8e34-f0d6e348dddc admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:15.328670 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a4868-94b1-497d-a6a4-7ed1b54574eb admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:15.340710 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c1d6e1-7822-417f-8e34-f0d6e348dddc admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:15.344764 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 801/1596] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:15.345944 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a4868-94b1-497d-a6a4-7ed1b54574eb admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:15.351495 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd8e4c-5a27-4992-802f-3eab59c089de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:15.353576 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd8e4c-5a27-4992-802f-3eab59c089de admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:15.353576 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd8e4c-5a27-4992-802f-3eab59c089de admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:15.354189 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd8e4c-5a27-4992-802f-3eab59c089de admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:15.355770 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd8e4c-5a27-4992-802f-3eab59c089de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:15.358689 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd8e4c-5a27-4992-802f-3eab59c089de admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:15.359367 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a4868-94b1-497d-a6a4-7ed1b54574eb admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:15.367071 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b05a08-6bac-418b-a38b-a2c2e8df4b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:15.368453 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b05a08-6bac-418b-a38b-a2c2e8df4b61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:15.368733 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b05a08-6bac-418b-a38b-a2c2e8df4b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:15.369078 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b05a08-6bac-418b-a38b-a2c2e8df4b61 admin admin] PATH_INFO: `/v3/projects/e85f3e6c18ae4466a1dc3e6464aa18e6/users/a3e93dfa85244133b1641a431994b47f/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:15.377981 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 796/1597] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:15.379416 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd8e4c-5a27-4992-802f-3eab59c089de admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:15.392973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b05a08-6bac-418b-a38b-a2c2e8df4b61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e85f3e6c18ae4466a1dc3e6464aa18e6, user_id=a3e93dfa85244133b1641a431994b47f, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:15.394166 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: WARNING py.warnings [None req-a2b05a08-6bac-418b-a38b-a2c2e8df4b61 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:08:15.394166 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: warnings.warn(msg) Aug 01 14:08:15.394166 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: Aug 01 14:08:15.395792 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd8e4c-5a27-4992-802f-3eab59c089de admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:15.397433 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b05a08-6bac-418b-a38b-a2c2e8df4b61 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:15.405526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4223fdd-efc2-4848-8ad1-5065e24ad024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:15.409235 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4223fdd-efc2-4848-8ad1-5065e24ad024 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:15.411284 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4223fdd-efc2-4848-8ad1-5065e24ad024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:15.411703 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4223fdd-efc2-4848-8ad1-5065e24ad024 admin admin] PATH_INFO: `/v3/projects/075e439a5de54261b5357fe8c513e9ee/users/ed6e0b24487342c1a9246579c0854653/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:15.436591 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 802/1598] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:15 2022] PUT /identity/v3/projects/e85f3e6c18ae4466a1dc3e6464aa18e6/users/a3e93dfa85244133b1641a431994b47f/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:15.436591 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b05a08-6bac-418b-a38b-a2c2e8df4b61 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:15.453321 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4223fdd-efc2-4848-8ad1-5065e24ad024 admin admin] RBAC: Authorizing `identity:create_grant(project_id=075e439a5de54261b5357fe8c513e9ee, user_id=ed6e0b24487342c1a9246579c0854653, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:15.458027 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4223fdd-efc2-4848-8ad1-5065e24ad024 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:15.490015 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 797/1599] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:15 2022] PUT /identity/v3/projects/075e439a5de54261b5357fe8c513e9ee/users/ed6e0b24487342c1a9246579c0854653/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:15.490015 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae607b6-5d32-424e-9921-9a22d3a7550a None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:15.490015 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae607b6-5d32-424e-9921-9a22d3a7550a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:15.490015 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae607b6-5d32-424e-9921-9a22d3a7550a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:15.520524 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:15.538340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-cae607b6-5d32-424e-9921-9a22d3a7550a None None] MFA Rules not processed for user `391bb00d7b25489985cbd5983ac41a19`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:15.540155 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-cae607b6-5d32-424e-9921-9a22d3a7550a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:15.560286 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b05a08-6bac-418b-a38b-a2c2e8df4b61 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:15.631957 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 798/1600] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:15.633351 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae607b6-5d32-424e-9921-9a22d3a7550a None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:15.735750 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:15.738322 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f1fca-705f-4109-ada7-f88eb6c4e89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:15.738322 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f1fca-705f-4109-ada7-f88eb6c4e89a admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:15.738322 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f1fca-705f-4109-ada7-f88eb6c4e89a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:15.738322 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f1fca-705f-4109-ada7-f88eb6c4e89a admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:15.738832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1f1fca-705f-4109-ada7-f88eb6c4e89a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:15.738832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1f1fca-705f-4109-ada7-f88eb6c4e89a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:15.738832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 803/1601] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:15.738832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f1fca-705f-4109-ada7-f88eb6c4e89a admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:15.738832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f1fca-705f-4109-ada7-f88eb6c4e89a admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:15.739256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e028a5f2-5764-4111-98d0-42fbcf72e8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:15.739256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028a5f2-5764-4111-98d0-42fbcf72e8df admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:15.739256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028a5f2-5764-4111-98d0-42fbcf72e8df admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:15.739256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028a5f2-5764-4111-98d0-42fbcf72e8df admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:15.739604 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e028a5f2-5764-4111-98d0-42fbcf72e8df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:15.739604 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e028a5f2-5764-4111-98d0-42fbcf72e8df admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:15.748475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 804/1602] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:15.749501 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e028a5f2-5764-4111-98d0-42fbcf72e8df admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:15.752519 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae607b6-5d32-424e-9921-9a22d3a7550a None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:15.759487 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b9a68-3c66-4700-83e4-88c105f53ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:15.760337 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b9a68-3c66-4700-83e4-88c105f53ff6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:15.760620 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b9a68-3c66-4700-83e4-88c105f53ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:15.760966 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b9a68-3c66-4700-83e4-88c105f53ff6 admin admin] PATH_INFO: `/v3/projects/a72b3bf493df4e3a92824bd41c12c6e7/users/a7dd1ba60fb64c18ba5f199d8e27d167/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:15.763737 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e028a5f2-5764-4111-98d0-42fbcf72e8df admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:15.769088 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb368f88-e8be-4a5b-a5e1-2fde50c8b622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:15.769959 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb368f88-e8be-4a5b-a5e1-2fde50c8b622 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:15.770386 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb368f88-e8be-4a5b-a5e1-2fde50c8b622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:15.770765 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb368f88-e8be-4a5b-a5e1-2fde50c8b622 admin admin] PATH_INFO: `/v3/projects/f209a9617cbe40f394023a7a1ba12b81/users/83360ac42cdc4dbabb3016177e523803/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:15.782436 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b9a68-3c66-4700-83e4-88c105f53ff6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a72b3bf493df4e3a92824bd41c12c6e7, user_id=a7dd1ba60fb64c18ba5f199d8e27d167, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:15.786073 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b9a68-3c66-4700-83e4-88c105f53ff6 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:15.791274 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb368f88-e8be-4a5b-a5e1-2fde50c8b622 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f209a9617cbe40f394023a7a1ba12b81, user_id=83360ac42cdc4dbabb3016177e523803, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:15.797858 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb368f88-e8be-4a5b-a5e1-2fde50c8b622 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:15.809003 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 799/1603] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:15 2022] PUT /identity/v3/projects/a72b3bf493df4e3a92824bd41c12c6e7/users/a7dd1ba60fb64c18ba5f199d8e27d167/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:15.810950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797135a2-9c02-461e-9790-ced05e4a8fb1 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:15.811380 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797135a2-9c02-461e-9790-ced05e4a8fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:15.811811 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797135a2-9c02-461e-9790-ced05e4a8fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:15.828898 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 805/1604] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:15 2022] PUT /identity/v3/projects/f209a9617cbe40f394023a7a1ba12b81/users/83360ac42cdc4dbabb3016177e523803/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:15.834285 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76478e-1d31-4c9d-8357-0b3deef39557 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:15.834705 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76478e-1d31-4c9d-8357-0b3deef39557 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:15.835386 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76478e-1d31-4c9d-8357-0b3deef39557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:15.856036 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-797135a2-9c02-461e-9790-ced05e4a8fb1 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:15.857916 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-797135a2-9c02-461e-9790-ced05e4a8fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:15.858905 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-ca76478e-1d31-4c9d-8357-0b3deef39557 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:15.860402 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ca76478e-1d31-4c9d-8357-0b3deef39557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:15.950438 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 806/1605] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:15.951835 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca76478e-1d31-4c9d-8357-0b3deef39557 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:15.952263 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 800/1606] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:15.956294 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797135a2-9c02-461e-9790-ced05e4a8fb1 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:15.967681 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797135a2-9c02-461e-9790-ced05e4a8fb1 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:15.977304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af80b2d-4685-403a-8a5c-8eff0227ce61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:15.978025 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af80b2d-4685-403a-8a5c-8eff0227ce61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:15.978375 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af80b2d-4685-403a-8a5c-8eff0227ce61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:15.978659 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af80b2d-4685-403a-8a5c-8eff0227ce61 admin admin] PATH_INFO: `/v3/projects/65fba8ff6b34466b8c456924d9cae36f/users/ba5780f4e0374edea061f6f2af11002c/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:15.980468 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca76478e-1d31-4c9d-8357-0b3deef39557 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:15.986401 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63019af0-babd-4d6d-83e2-5ebf4995d87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:15.987140 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63019af0-babd-4d6d-83e2-5ebf4995d87c admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:15.987426 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63019af0-babd-4d6d-83e2-5ebf4995d87c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:15.987697 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63019af0-babd-4d6d-83e2-5ebf4995d87c admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:15.988397 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63019af0-babd-4d6d-83e2-5ebf4995d87c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:15.990328 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63019af0-babd-4d6d-83e2-5ebf4995d87c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:15.995732 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af80b2d-4685-403a-8a5c-8eff0227ce61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65fba8ff6b34466b8c456924d9cae36f, user_id=ba5780f4e0374edea061f6f2af11002c, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:16.000315 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af80b2d-4685-403a-8a5c-8eff0227ce61 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:16.018087 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 801/1607] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:15 2022] PUT /identity/v3/projects/65fba8ff6b34466b8c456924d9cae36f/users/ba5780f4e0374edea061f6f2af11002c/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:16.019417 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af80b2d-4685-403a-8a5c-8eff0227ce61 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:16.022513 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 807/1608] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:15 2022] POST /identity/v3/projects => generated 403 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:16.023894 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43af10b-7b89-42f5-a883-2df7ab308961 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:16.024179 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43af10b-7b89-42f5-a883-2df7ab308961 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:16.024495 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43af10b-7b89-42f5-a883-2df7ab308961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:16.075827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a43af10b-7b89-42f5-a883-2df7ab308961 None None] MFA Rules not processed for user `a7dd1ba60fb64c18ba5f199d8e27d167`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:16.078063 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a43af10b-7b89-42f5-a883-2df7ab308961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:16.100701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:16.107772 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af80b2d-4685-403a-8a5c-8eff0227ce61 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:16.120089 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:16.174669 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 808/1609] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:16.175986 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec5428-e3ba-4e2a-bfa8-7f4f362f71c4 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:16.176969 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec5428-e3ba-4e2a-bfa8-7f4f362f71c4 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:16.176969 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec5428-e3ba-4e2a-bfa8-7f4f362f71c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:16.187296 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1d27e-e3fb-4c8a-9428-633387a60ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:16.188105 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1d27e-e3fb-4c8a-9428-633387a60ba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:16.188493 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1d27e-e3fb-4c8a-9428-633387a60ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:16.188869 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1d27e-e3fb-4c8a-9428-633387a60ba5 admin admin] PATH_INFO: `/v3/projects/34cf7d36f6ac4ad6bc926527920c32b6/users/b0175d8bed8846c4a3350bc0023b5bb4/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:16.205409 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf1d27e-e3fb-4c8a-9428-633387a60ba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34cf7d36f6ac4ad6bc926527920c32b6, user_id=b0175d8bed8846c4a3350bc0023b5bb4, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:16.208459 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf1d27e-e3fb-4c8a-9428-633387a60ba5 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:16.210116 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a8ec5428-e3ba-4e2a-bfa8-7f4f362f71c4 None None] MFA Rules not processed for user `83360ac42cdc4dbabb3016177e523803`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:16.215485 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a8ec5428-e3ba-4e2a-bfa8-7f4f362f71c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:16.227625 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 802/1610] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:16 2022] PUT /identity/v3/projects/34cf7d36f6ac4ad6bc926527920c32b6/users/b0175d8bed8846c4a3350bc0023b5bb4/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:16.228873 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1d27e-e3fb-4c8a-9428-633387a60ba5 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:16.333455 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1d27e-e3fb-4c8a-9428-633387a60ba5 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:16.339926 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 809/1611] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:16.341650 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdde424-5e8d-425a-9917-e42f5f0d8112 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:16.342313 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdde424-5e8d-425a-9917-e42f5f0d8112 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:16.342797 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdde424-5e8d-425a-9917-e42f5f0d8112 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:16.343085 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdde424-5e8d-425a-9917-e42f5f0d8112 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:16.347177 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec5428-e3ba-4e2a-bfa8-7f4f362f71c4 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:16.348742 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a8ec5428-e3ba-4e2a-bfa8-7f4f362f71c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:16.430202 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec5428-e3ba-4e2a-bfa8-7f4f362f71c4 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:16.435530 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdde424-5e8d-425a-9917-e42f5f0d8112 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:16.435755 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c3f75-4bc0-4204-a22a-25d38c1a8051 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:16.436490 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c3f75-4bc0-4204-a22a-25d38c1a8051 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:16.436771 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c3f75-4bc0-4204-a22a-25d38c1a8051 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:16.437193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c3f75-4bc0-4204-a22a-25d38c1a8051 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:16.437615 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdde424-5e8d-425a-9917-e42f5f0d8112 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:16.447967 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51c3f75-4bc0-4204-a22a-25d38c1a8051 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:16.449671 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51c3f75-4bc0-4204-a22a-25d38c1a8051 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:16.450383 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 803/1612] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:16.461987 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 810/1613] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:16.463021 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c0271-8d77-4ae1-aaee-5774fc2da6c6 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:16.463295 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c0271-8d77-4ae1-aaee-5774fc2da6c6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:16.463566 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c0271-8d77-4ae1-aaee-5774fc2da6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:16.465398 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdde424-5e8d-425a-9917-e42f5f0d8112 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:16.489616 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdde424-5e8d-425a-9917-e42f5f0d8112 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:16.506416 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b84c2b-0805-4b67-9d0d-d19f068b0d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:16.510055 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-401c0271-8d77-4ae1-aaee-5774fc2da6c6 None None] MFA Rules not processed for user `ba5780f4e0374edea061f6f2af11002c`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:16.510387 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b84c2b-0805-4b67-9d0d-d19f068b0d93 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:16.510801 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b84c2b-0805-4b67-9d0d-d19f068b0d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:16.511175 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b84c2b-0805-4b67-9d0d-d19f068b0d93 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:16.511967 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-401c0271-8d77-4ae1-aaee-5774fc2da6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:16.512511 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b84c2b-0805-4b67-9d0d-d19f068b0d93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:16.518191 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b84c2b-0805-4b67-9d0d-d19f068b0d93 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:16.572927 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 804/1614] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:16 2022] POST /identity/v3/users => generated 308 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:16.574116 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42252a-e1fa-4c6e-b0b5-7110b391404b None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:16.574551 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42252a-e1fa-4c6e-b0b5-7110b391404b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:16.574748 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42252a-e1fa-4c6e-b0b5-7110b391404b None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:16.580210 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 805/1615] 10.222.0.45 () {54 vars in 962 bytes} [Mon Aug 1 14:08:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:08:16.583878 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e090b2d-98e9-4a92-8954-3a5074a77d9b None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:16.584380 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e090b2d-98e9-4a92-8954-3a5074a77d9b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:16.584658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e090b2d-98e9-4a92-8954-3a5074a77d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:16.597839 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:16.657194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-5e090b2d-98e9-4a92-8954-3a5074a77d9b None None] MFA Rules not processed for user `b0175d8bed8846c4a3350bc0023b5bb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:16.662087 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5e090b2d-98e9-4a92-8954-3a5074a77d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:16.664936 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 811/1616] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:16.667248 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59f722-184c-4567-ac1c-1dbca803eb18 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:16.667870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59f722-184c-4567-ac1c-1dbca803eb18 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:16.668300 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59f722-184c-4567-ac1c-1dbca803eb18 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:16.670856 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 812/1617] 10.222.0.45 () {54 vars in 962 bytes} [Mon Aug 1 14:08:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:08:16.671742 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c59f722-184c-4567-ac1c-1dbca803eb18 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:16.692157 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c59f722-184c-4567-ac1c-1dbca803eb18 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:16.703748 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:16.706808 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08461705-b424-446e-ba3b-d970d7585df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:16.711354 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08461705-b424-446e-ba3b-d970d7585df4 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:16.713523 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08461705-b424-446e-ba3b-d970d7585df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:16.713933 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08461705-b424-446e-ba3b-d970d7585df4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:16.716717 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08461705-b424-446e-ba3b-d970d7585df4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:16.722360 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08461705-b424-446e-ba3b-d970d7585df4 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:16.752816 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 813/1618] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:16 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:16.755578 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65717f34-e0a6-4e78-87f2-f0f7944baadd None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:16.756457 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65717f34-e0a6-4e78-87f2-f0f7944baadd None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:16.756768 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65717f34-e0a6-4e78-87f2-f0f7944baadd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:16.803657 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 806/1619] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:16.805277 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ed92e-b183-40bf-bb49-ede15c93798c None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:16.805794 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ed92e-b183-40bf-bb49-ede15c93798c None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:16.806298 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ed92e-b183-40bf-bb49-ede15c93798c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:16.850821 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-65717f34-e0a6-4e78-87f2-f0f7944baadd None None] MFA Rules not processed for user `5b3adc14dcaf4ac79b43bb97fa1ba1d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:16.853589 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-65717f34-e0a6-4e78-87f2-f0f7944baadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:16.871899 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-dc1ed92e-b183-40bf-bb49-ede15c93798c None None] MFA Rules not processed for user `5b3adc14dcaf4ac79b43bb97fa1ba1d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:16.875589 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-dc1ed92e-b183-40bf-bb49-ede15c93798c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:16.970104 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 814/1620] 10.222.0.45 () {58 vars in 1059 bytes} [Mon Aug 1 14:08:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:16.972002 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65717f34-e0a6-4e78-87f2-f0f7944baadd None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:16.986472 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65717f34-e0a6-4e78-87f2-f0f7944baadd None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:16.996079 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75636e92-3806-414e-9316-26512e639170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:17.002264 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75636e92-3806-414e-9316-26512e639170 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:17.002729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75636e92-3806-414e-9316-26512e639170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:17.003242 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75636e92-3806-414e-9316-26512e639170 admin admin] PATH_INFO: `/v3/projects/6ce580178de04321a708a358cf4af8a5/users/af8181cc5e16451ba94e9f4c9c7ca629/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:17.035730 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75636e92-3806-414e-9316-26512e639170 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ce580178de04321a708a358cf4af8a5, user_id=af8181cc5e16451ba94e9f4c9c7ca629, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:17.040002 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75636e92-3806-414e-9316-26512e639170 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:17.061112 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 815/1621] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:16 2022] PUT /identity/v3/projects/6ce580178de04321a708a358cf4af8a5/users/af8181cc5e16451ba94e9f4c9c7ca629/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:17.062327 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75636e92-3806-414e-9316-26512e639170 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:17.114572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 807/1622] 10.222.0.45 () {58 vars in 1059 bytes} [Mon Aug 1 14:08:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:17.116257 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ed92e-b183-40bf-bb49-ede15c93798c None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:17.157217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ed92e-b183-40bf-bb49-ede15c93798c None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:17.170735 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75636e92-3806-414e-9316-26512e639170 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:17.175876 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b72ca-1224-482c-94b3-e6e0cdd1ed0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:17.176695 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b72ca-1224-482c-94b3-e6e0cdd1ed0c admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:17.177075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b72ca-1224-482c-94b3-e6e0cdd1ed0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:17.178261 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b72ca-1224-482c-94b3-e6e0cdd1ed0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:17.179487 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26b72ca-1224-482c-94b3-e6e0cdd1ed0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:17.181875 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26b72ca-1224-482c-94b3-e6e0cdd1ed0c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:17.226799 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 816/1623] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:17 2022] POST /identity/v3/projects => generated 373 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:17.228009 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b72ca-1224-482c-94b3-e6e0cdd1ed0c admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:17.246315 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:17.352670 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9f30e2-268f-4a1d-941b-cbb2b140807c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:17.360308 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f30e2-268f-4a1d-941b-cbb2b140807c service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:17.360308 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f30e2-268f-4a1d-941b-cbb2b140807c service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:17.360308 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f30e2-268f-4a1d-941b-cbb2b140807c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:17.432321 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b72ca-1224-482c-94b3-e6e0cdd1ed0c admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:17.451476 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce00be0-ef15-4a94-ac91-c7f073d6bf45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:17.452187 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce00be0-ef15-4a94-ac91-c7f073d6bf45 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:17.452516 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce00be0-ef15-4a94-ac91-c7f073d6bf45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:17.452792 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce00be0-ef15-4a94-ac91-c7f073d6bf45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:17.455343 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce00be0-ef15-4a94-ac91-c7f073d6bf45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:17.458658 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce00be0-ef15-4a94-ac91-c7f073d6bf45 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:17.486072 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 817/1624] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:17.486968 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce00be0-ef15-4a94-ac91-c7f073d6bf45 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:17.514001 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:17.557138 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9f30e2-268f-4a1d-941b-cbb2b140807c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:17.558956 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9f30e2-268f-4a1d-941b-cbb2b140807c service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:17.604348 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 808/1625] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:17.607517 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9f30e2-268f-4a1d-941b-cbb2b140807c service nova] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:17.641792 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9f30e2-268f-4a1d-941b-cbb2b140807c service nova] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:17.655542 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc26d8-8f54-4590-bc2b-29439d48ae21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:17.656869 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc26d8-8f54-4590-bc2b-29439d48ae21 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:17.656869 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc26d8-8f54-4590-bc2b-29439d48ae21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:17.656869 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc26d8-8f54-4590-bc2b-29439d48ae21 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:17.657455 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc26d8-8f54-4590-bc2b-29439d48ae21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:17.664078 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc26d8-8f54-4590-bc2b-29439d48ae21 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:17.715392 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 809/1626] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:17 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:17.719684 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc26d8-8f54-4590-bc2b-29439d48ae21 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:17.729835 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce00be0-ef15-4a94-ac91-c7f073d6bf45 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:17.742632 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc26d8-8f54-4590-bc2b-29439d48ae21 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:17.744836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c319498f-3a27-424f-b324-508ece776dbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:17.745494 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319498f-3a27-424f-b324-508ece776dbd service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:17.745766 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319498f-3a27-424f-b324-508ece776dbd service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:17.746032 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319498f-3a27-424f-b324-508ece776dbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:17.763210 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d461760c-d5bb-4433-ba24-88d0cf31d7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:17.764518 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d461760c-d5bb-4433-ba24-88d0cf31d7ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:17.764746 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d461760c-d5bb-4433-ba24-88d0cf31d7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:17.765015 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d461760c-d5bb-4433-ba24-88d0cf31d7ea admin admin] PATH_INFO: `/v3/projects/6ce580178de04321a708a358cf4af8a5/users/af8181cc5e16451ba94e9f4c9c7ca629/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:17.769255 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d461760c-d5bb-4433-ba24-88d0cf31d7ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ce580178de04321a708a358cf4af8a5, user_id=af8181cc5e16451ba94e9f4c9c7ca629, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:17.776539 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d461760c-d5bb-4433-ba24-88d0cf31d7ea admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:17.807558 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 810/1627] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:17 2022] PUT /identity/v3/projects/6ce580178de04321a708a358cf4af8a5/users/af8181cc5e16451ba94e9f4c9c7ca629/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:17.812728 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d461760c-d5bb-4433-ba24-88d0cf31d7ea admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:17.936115 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c319498f-3a27-424f-b324-508ece776dbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:17.941435 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c319498f-3a27-424f-b324-508ece776dbd service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:17.971178 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 818/1628] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:17.972891 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf0f47-8c85-4dcf-9173-a3c1b1b79374 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:17.973073 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf0f47-8c85-4dcf-9173-a3c1b1b79374 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:17.973204 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf0f47-8c85-4dcf-9173-a3c1b1b79374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:18.044730 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d461760c-d5bb-4433-ba24-88d0cf31d7ea admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:18.070715 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6fca1b-2316-4669-9686-2ce138a18d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:18.071601 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6fca1b-2316-4669-9686-2ce138a18d9c admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:18.071885 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6fca1b-2316-4669-9686-2ce138a18d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:18.072158 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6fca1b-2316-4669-9686-2ce138a18d9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:18.072911 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6fca1b-2316-4669-9686-2ce138a18d9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:18.074612 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6fca1b-2316-4669-9686-2ce138a18d9c admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:18.088779 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-31bf0f47-8c85-4dcf-9173-a3c1b1b79374 None None] MFA Rules not processed for user `af8181cc5e16451ba94e9f4c9c7ca629`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:18.091883 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-31bf0f47-8c85-4dcf-9173-a3c1b1b79374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:18.119684 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 811/1629] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:18.123252 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6fca1b-2316-4669-9686-2ce138a18d9c admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:18.164430 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6fca1b-2316-4669-9686-2ce138a18d9c admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:18.182572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc291bd1-39fa-4c76-ad9c-5edceab712c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:18.183284 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc291bd1-39fa-4c76-ad9c-5edceab712c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:18.183559 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc291bd1-39fa-4c76-ad9c-5edceab712c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:18.183827 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc291bd1-39fa-4c76-ad9c-5edceab712c2 admin admin] PATH_INFO: `/v3/projects/f9a2e02f2cff49258df6de06399f8239/users/b911b435ebcf4f2b83cfd7b68589d73b/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:18.210371 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:18.249143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc291bd1-39fa-4c76-ad9c-5edceab712c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9a2e02f2cff49258df6de06399f8239, user_id=b911b435ebcf4f2b83cfd7b68589d73b, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:18.252828 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc291bd1-39fa-4c76-ad9c-5edceab712c2 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:18.280187 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 812/1630] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:18 2022] PUT /identity/v3/projects/f9a2e02f2cff49258df6de06399f8239/users/b911b435ebcf4f2b83cfd7b68589d73b/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:18.285308 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc291bd1-39fa-4c76-ad9c-5edceab712c2 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:18.353769 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 819/1631] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:17 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:18.374691 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62bc363-9c8f-4117-9c12-1db419ce8e37 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:18.375244 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62bc363-9c8f-4117-9c12-1db419ce8e37 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:18.375675 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62bc363-9c8f-4117-9c12-1db419ce8e37 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:18.377364 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 820/1632] 10.222.0.45 () {54 vars in 912 bytes} [Mon Aug 1 14:08:18 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:08:18.378589 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62bc363-9c8f-4117-9c12-1db419ce8e37 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:18.472513 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc291bd1-39fa-4c76-ad9c-5edceab712c2 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:18.478964 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8f8f3-9e9d-4576-80ca-0d5118691496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:18.480668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8f8f3-9e9d-4576-80ca-0d5118691496 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:18.480953 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8f8f3-9e9d-4576-80ca-0d5118691496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:18.481229 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8f8f3-9e9d-4576-80ca-0d5118691496 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:18.482137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c8f8f3-9e9d-4576-80ca-0d5118691496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:18.484161 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c8f8f3-9e9d-4576-80ca-0d5118691496 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:18.499934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 813/1633] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:18.502282 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdcf962-e064-4306-9c5d-d632087edbd6 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:18.502598 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdcf962-e064-4306-9c5d-d632087edbd6 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:18.503044 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdcf962-e064-4306-9c5d-d632087edbd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:18.526018 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62bc363-9c8f-4117-9c12-1db419ce8e37 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:18.531738 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-ccdcf962-e064-4306-9c5d-d632087edbd6 None None] MFA Rules not processed for user `5b3adc14dcaf4ac79b43bb97fa1ba1d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:18.532341 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7596ee-bc0d-47bb-95cc-31179e1c824b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:18.533164 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ccdcf962-e064-4306-9c5d-d632087edbd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:18.537011 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7596ee-bc0d-47bb-95cc-31179e1c824b service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:18.537583 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7596ee-bc0d-47bb-95cc-31179e1c824b service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:18.537733 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7596ee-bc0d-47bb-95cc-31179e1c824b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:18.589732 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:18.671285 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7596ee-bc0d-47bb-95cc-31179e1c824b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:18.673415 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7596ee-bc0d-47bb-95cc-31179e1c824b service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:18.684462 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 814/1634] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Aug 1 14:08:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:18.686550 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdcf962-e064-4306-9c5d-d632087edbd6 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:18.704574 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 821/1635] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:18 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:18.706206 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182622ba-873e-4f07-97a7-dbec5752ffbc None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:18.708737 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182622ba-873e-4f07-97a7-dbec5752ffbc None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:18.713404 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182622ba-873e-4f07-97a7-dbec5752ffbc None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:18.717726 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 822/1636] 10.222.0.45 () {54 vars in 912 bytes} [Mon Aug 1 14:08:18 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:08:18.718883 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182622ba-873e-4f07-97a7-dbec5752ffbc None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:18.724545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdcf962-e064-4306-9c5d-d632087edbd6 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:18.742227 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb943d20-6242-4378-8b5c-82aeb4fa51f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:18.743083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb943d20-6242-4378-8b5c-82aeb4fa51f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:18.743378 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb943d20-6242-4378-8b5c-82aeb4fa51f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:18.743655 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb943d20-6242-4378-8b5c-82aeb4fa51f5 admin admin] PATH_INFO: `/v3/projects/f9a2e02f2cff49258df6de06399f8239/users/b911b435ebcf4f2b83cfd7b68589d73b/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:18.746987 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182622ba-873e-4f07-97a7-dbec5752ffbc None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:18.751821 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13438678-a232-435a-9d6a-f38202ec2a83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:18.752796 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13438678-a232-435a-9d6a-f38202ec2a83 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:18.753136 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13438678-a232-435a-9d6a-f38202ec2a83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:18.753416 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13438678-a232-435a-9d6a-f38202ec2a83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:18.757244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb943d20-6242-4378-8b5c-82aeb4fa51f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9a2e02f2cff49258df6de06399f8239, user_id=b911b435ebcf4f2b83cfd7b68589d73b, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:18.760165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb943d20-6242-4378-8b5c-82aeb4fa51f5 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:18.768562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13438678-a232-435a-9d6a-f38202ec2a83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:18.768562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13438678-a232-435a-9d6a-f38202ec2a83 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:18.796758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 823/1637] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:18.799029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba42f50-fa56-411c-8b31-6e38443a0e12 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:18.799326 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba42f50-fa56-411c-8b31-6e38443a0e12 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:18.799680 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba42f50-fa56-411c-8b31-6e38443a0e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:18.814105 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 815/1638] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:18 2022] PUT /identity/v3/projects/f9a2e02f2cff49258df6de06399f8239/users/b911b435ebcf4f2b83cfd7b68589d73b/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:18.835948 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16bba7-7ecf-481f-a461-47d48b5a66ee None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:18.836580 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16bba7-7ecf-481f-a461-47d48b5a66ee None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:18.837444 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16bba7-7ecf-481f-a461-47d48b5a66ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:18.859476 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8ba42f50-fa56-411c-8b31-6e38443a0e12 None None] MFA Rules not processed for user `5b3adc14dcaf4ac79b43bb97fa1ba1d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:18.861463 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8ba42f50-fa56-411c-8b31-6e38443a0e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:18.898182 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-3e16bba7-7ecf-481f-a461-47d48b5a66ee None None] MFA Rules not processed for user `b911b435ebcf4f2b83cfd7b68589d73b`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:18.900026 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3e16bba7-7ecf-481f-a461-47d48b5a66ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:19.082936 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:19.104904 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 816/1639] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:18 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:08:19.118251 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 824/1640] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Aug 1 14:08:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:19.145708 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16bba7-7ecf-481f-a461-47d48b5a66ee None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:19.151014 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba42f50-fa56-411c-8b31-6e38443a0e12 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:19.269564 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba42f50-fa56-411c-8b31-6e38443a0e12 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:19.281573 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50935e32-c206-4859-ae0c-f4f886a9353f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:19.282261 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50935e32-c206-4859-ae0c-f4f886a9353f service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:19.282587 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50935e32-c206-4859-ae0c-f4f886a9353f service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:19.282899 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50935e32-c206-4859-ae0c-f4f886a9353f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:19.307240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50935e32-c206-4859-ae0c-f4f886a9353f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:19.309943 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50935e32-c206-4859-ae0c-f4f886a9353f service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:19.341924 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16bba7-7ecf-481f-a461-47d48b5a66ee None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:19.342490 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 825/1641] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:19 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:19.348311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5aa3ae-f262-407f-ad6d-616e1040f645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:19.348982 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5aa3ae-f262-407f-ad6d-616e1040f645 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:19.349268 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5aa3ae-f262-407f-ad6d-616e1040f645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:19.349578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5aa3ae-f262-407f-ad6d-616e1040f645 admin admin] PATH_INFO: `/v3/users/391bb00d7b25489985cbd5983ac41a19` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:19.351015 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5aa3ae-f262-407f-ad6d-616e1040f645 admin admin] RBAC: Authorizing `identity:delete_user(user_id=391bb00d7b25489985cbd5983ac41a19)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:19.353742 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5aa3ae-f262-407f-ad6d-616e1040f645 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:19.409509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:19.452246 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ea5aa3ae-f262-407f-ad6d-616e1040f645 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '391bb00d7b25489985cbd5983ac41a19'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:19.452246 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ea5aa3ae-f262-407f-ad6d-616e1040f645 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '391bb00d7b25489985cbd5983ac41a19'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:19.458659 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ea5aa3ae-f262-407f-ad6d-616e1040f645 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '391bb00d7b25489985cbd5983ac41a19'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:19.476336 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ea5aa3ae-f262-407f-ad6d-616e1040f645 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '391bb00d7b25489985cbd5983ac41a19'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:19.485766 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50935e32-c206-4859-ae0c-f4f886a9353f service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:19.503365 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50935e32-c206-4859-ae0c-f4f886a9353f service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:19.514791 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5632a-924c-4a90-9eea-ba34a8d5b4d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:19.515971 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5632a-924c-4a90-9eea-ba34a8d5b4d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:19.518805 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5632a-924c-4a90-9eea-ba34a8d5b4d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:19.520051 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5632a-924c-4a90-9eea-ba34a8d5b4d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:19.524768 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5cd5632a-924c-4a90-9eea-ba34a8d5b4d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:19.528291 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ea5aa3ae-f262-407f-ad6d-616e1040f645 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '391bb00d7b25489985cbd5983ac41a19'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:19.560718 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 817/1642] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:08:19 2022] DELETE /identity/v3/users/391bb00d7b25489985cbd5983ac41a19 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:19.566467 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bde2b0-f20e-491a-b23a-066ebd8b7868 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:19.566757 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bde2b0-f20e-491a-b23a-066ebd8b7868 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:19.568120 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bde2b0-f20e-491a-b23a-066ebd8b7868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:19.585648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5632a-924c-4a90-9eea-ba34a8d5b4d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:19.591675 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5632a-924c-4a90-9eea-ba34a8d5b4d0 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:19.625769 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-b8bde2b0-f20e-491a-b23a-066ebd8b7868 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:19.627880 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 826/1643] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:19.628995 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b8bde2b0-f20e-491a-b23a-066ebd8b7868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:19.736147 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 818/1644] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:19.773102 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5632a-924c-4a90-9eea-ba34a8d5b4d0 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:19.775475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5cd5632a-924c-4a90-9eea-ba34a8d5b4d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:19.997087 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5632a-924c-4a90-9eea-ba34a8d5b4d0 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:20.166623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1a569a-79b6-4f29-9777-041a22d36d57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:20.167587 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1a569a-79b6-4f29-9777-041a22d36d57 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:20.167969 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1a569a-79b6-4f29-9777-041a22d36d57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:20.168387 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1a569a-79b6-4f29-9777-041a22d36d57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:20.218334 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bde2b0-f20e-491a-b23a-066ebd8b7868 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:20.220501 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b8bde2b0-f20e-491a-b23a-066ebd8b7868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:20.223808 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1a569a-79b6-4f29-9777-041a22d36d57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:20.227623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1a569a-79b6-4f29-9777-041a22d36d57 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:20.248648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 827/1645] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:20.250061 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1a569a-79b6-4f29-9777-041a22d36d57 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:20.252019 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-bf1a569a-79b6-4f29-9777-041a22d36d57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:20.385638 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:20.474888 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1a569a-79b6-4f29-9777-041a22d36d57 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:20.489959 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af75dbc-3b6c-44dd-b4b3-d4beaca4549b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:20.490872 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af75dbc-3b6c-44dd-b4b3-d4beaca4549b service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:20.491234 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af75dbc-3b6c-44dd-b4b3-d4beaca4549b service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:20.491583 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af75dbc-3b6c-44dd-b4b3-d4beaca4549b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:20.495404 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6af75dbc-3b6c-44dd-b4b3-d4beaca4549b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:20.521698 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bde2b0-f20e-491a-b23a-066ebd8b7868 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:20.548998 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb73e5-7ed3-420f-a371-d31c454c490f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:20.555584 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb73e5-7ed3-420f-a371-d31c454c490f service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:20.558160 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb73e5-7ed3-420f-a371-d31c454c490f service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:20.609775 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb73e5-7ed3-420f-a371-d31c454c490f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:20.609775 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-69cb73e5-7ed3-420f-a371-d31c454c490f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:20.733159 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af75dbc-3b6c-44dd-b4b3-d4beaca4549b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:20.735807 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af75dbc-3b6c-44dd-b4b3-d4beaca4549b service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:20.781164 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:20.784797 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 828/1646] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:20.819750 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cb73e5-7ed3-420f-a371-d31c454c490f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:20.821383 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cb73e5-7ed3-420f-a371-d31c454c490f service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:20.847638 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 819/1647] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 629 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:08:21.601773 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af75dbc-3b6c-44dd-b4b3-d4beaca4549b service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:21.604176 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6af75dbc-3b6c-44dd-b4b3-d4beaca4549b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:21.672143 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af75dbc-3b6c-44dd-b4b3-d4beaca4549b service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:21.725777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421760c-9d62-4ed3-b421-d5f533c8e2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:21.731327 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421760c-9d62-4ed3-b421-d5f533c8e2f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:21.731773 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421760c-9d62-4ed3-b421-d5f533c8e2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:21.734782 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421760c-9d62-4ed3-b421-d5f533c8e2f5 admin admin] PATH_INFO: `/v3/users/ba5780f4e0374edea061f6f2af11002c` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:21.745041 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421760c-9d62-4ed3-b421-d5f533c8e2f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba5780f4e0374edea061f6f2af11002c)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:21.746818 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: WARNING py.warnings [None req-2421760c-9d62-4ed3-b421-d5f533c8e2f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:08:21.746818 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: warnings.warn(msg) Aug 01 14:08:21.746818 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: Aug 01 14:08:21.757550 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421760c-9d62-4ed3-b421-d5f533c8e2f5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:22.018006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2421760c-9d62-4ed3-b421-d5f533c8e2f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba5780f4e0374edea061f6f2af11002c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:22.019191 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2421760c-9d62-4ed3-b421-d5f533c8e2f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba5780f4e0374edea061f6f2af11002c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:22.086647 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2421760c-9d62-4ed3-b421-d5f533c8e2f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba5780f4e0374edea061f6f2af11002c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:22.108794 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2421760c-9d62-4ed3-b421-d5f533c8e2f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba5780f4e0374edea061f6f2af11002c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:22.111805 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2421760c-9d62-4ed3-b421-d5f533c8e2f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba5780f4e0374edea061f6f2af11002c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:22.141918 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 829/1648] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:08:21 2022] DELETE /identity/v3/users/ba5780f4e0374edea061f6f2af11002c => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:08:22.146566 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac9373-7aa8-485e-9779-3841f9dfbb11 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:22.148012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac9373-7aa8-485e-9779-3841f9dfbb11 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:22.148497 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac9373-7aa8-485e-9779-3841f9dfbb11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:22.193387 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-99ac9373-7aa8-485e-9779-3841f9dfbb11 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:22.194652 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-99ac9373-7aa8-485e-9779-3841f9dfbb11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:22.367078 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 820/1649] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:22.401433 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421760c-9d62-4ed3-b421-d5f533c8e2f5 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:22.403677 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2421760c-9d62-4ed3-b421-d5f533c8e2f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:22.567114 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421760c-9d62-4ed3-b421-d5f533c8e2f5 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:22.580703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96a250-aa77-4608-a454-ef798a078b24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:22.581726 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96a250-aa77-4608-a454-ef798a078b24 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:22.582255 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96a250-aa77-4608-a454-ef798a078b24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:22.582799 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96a250-aa77-4608-a454-ef798a078b24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:22.604593 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad96a250-aa77-4608-a454-ef798a078b24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:22.606807 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad96a250-aa77-4608-a454-ef798a078b24 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:22.639585 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 830/1650] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:22.718224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac9373-7aa8-485e-9779-3841f9dfbb11 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:22.722194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-99ac9373-7aa8-485e-9779-3841f9dfbb11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:22.744474 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac9373-7aa8-485e-9779-3841f9dfbb11 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:22.760911 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d399b7-e515-4824-bf6b-bc618d112697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:22.761175 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d399b7-e515-4824-bf6b-bc618d112697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:22.761488 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d399b7-e515-4824-bf6b-bc618d112697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:22.761897 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d399b7-e515-4824-bf6b-bc618d112697 admin admin] PATH_INFO: `/v3/projects/77b5cb16a7fe46a68be4a6c47f234e36` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:22.765400 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d399b7-e515-4824-bf6b-bc618d112697 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77b5cb16a7fe46a68be4a6c47f234e36)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:22.766720 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: WARNING py.warnings [None req-c8d399b7-e515-4824-bf6b-bc618d112697 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:08:22.766720 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: warnings.warn(msg) Aug 01 14:08:22.766720 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: Aug 01 14:08:22.768900 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d399b7-e515-4824-bf6b-bc618d112697 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:22.918616 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-c8d399b7-e515-4824-bf6b-bc618d112697 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77b5cb16a7fe46a68be4a6c47f234e36', 'request_id': 'req-c8d399b7-e515-4824-bf6b-bc618d112697'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:22.961810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-c8d399b7-e515-4824-bf6b-bc618d112697 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77b5cb16a7fe46a68be4a6c47f234e36', 'request_id': 'req-c8d399b7-e515-4824-bf6b-bc618d112697'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:22.981146 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-c8d399b7-e515-4824-bf6b-bc618d112697 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77b5cb16a7fe46a68be4a6c47f234e36', 'request_id': 'req-c8d399b7-e515-4824-bf6b-bc618d112697'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:23.006715 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-c8d399b7-e515-4824-bf6b-bc618d112697 admin admin] The token cache is being invalidate because project 77b5cb16a7fe46a68be4a6c47f234e36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:08:23.007470 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-c8d399b7-e515-4824-bf6b-bc618d112697 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:23.008530 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-c8d399b7-e515-4824-bf6b-bc618d112697 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:23.012292 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 821/1651] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:08:22 2022] DELETE /identity/v3/projects/77b5cb16a7fe46a68be4a6c47f234e36 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:23.016954 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96a250-aa77-4608-a454-ef798a078b24 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:23.036028 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96a250-aa77-4608-a454-ef798a078b24 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:23.076501 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:23.314456 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea39683-3998-42a3-b482-6081d0723f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:23.315187 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea39683-3998-42a3-b482-6081d0723f16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:23.315477 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea39683-3998-42a3-b482-6081d0723f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:23.315756 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea39683-3998-42a3-b482-6081d0723f16 admin admin] PATH_INFO: `/v3/users/af8181cc5e16451ba94e9f4c9c7ca629` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:23.320887 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea39683-3998-42a3-b482-6081d0723f16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af8181cc5e16451ba94e9f4c9c7ca629)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:23.328303 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea39683-3998-42a3-b482-6081d0723f16 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:23.420241 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d399b7-e515-4824-bf6b-bc618d112697 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:23.422119 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c8d399b7-e515-4824-bf6b-bc618d112697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:23.453030 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d399b7-e515-4824-bf6b-bc618d112697 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:23.469437 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a2042-a2ff-4d9a-8a9c-af32777c568d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:23.470679 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a2042-a2ff-4d9a-8a9c-af32777c568d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:23.471070 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a2042-a2ff-4d9a-8a9c-af32777c568d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:23.471513 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a2042-a2ff-4d9a-8a9c-af32777c568d admin admin] PATH_INFO: `/v3/projects/65fba8ff6b34466b8c456924d9cae36f` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:23.473071 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a2042-a2ff-4d9a-8a9c-af32777c568d admin admin] RBAC: Authorizing `identity:delete_project(project_id=65fba8ff6b34466b8c456924d9cae36f)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:23.475211 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a2042-a2ff-4d9a-8a9c-af32777c568d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:23.548887 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-aea39683-3998-42a3-b482-6081d0723f16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af8181cc5e16451ba94e9f4c9c7ca629'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:23.552050 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-aea39683-3998-42a3-b482-6081d0723f16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af8181cc5e16451ba94e9f4c9c7ca629'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:23.611087 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-2b0a2042-a2ff-4d9a-8a9c-af32777c568d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65fba8ff6b34466b8c456924d9cae36f', 'request_id': 'req-2b0a2042-a2ff-4d9a-8a9c-af32777c568d'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:23.633809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-2b0a2042-a2ff-4d9a-8a9c-af32777c568d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65fba8ff6b34466b8c456924d9cae36f', 'request_id': 'req-2b0a2042-a2ff-4d9a-8a9c-af32777c568d'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:23.634079 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-aea39683-3998-42a3-b482-6081d0723f16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af8181cc5e16451ba94e9f4c9c7ca629'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:23.659115 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-2b0a2042-a2ff-4d9a-8a9c-af32777c568d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65fba8ff6b34466b8c456924d9cae36f', 'request_id': 'req-2b0a2042-a2ff-4d9a-8a9c-af32777c568d'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:23.664470 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:23.664629 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-aea39683-3998-42a3-b482-6081d0723f16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af8181cc5e16451ba94e9f4c9c7ca629'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:23.668041 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-aea39683-3998-42a3-b482-6081d0723f16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af8181cc5e16451ba94e9f4c9c7ca629'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:23.682999 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-2b0a2042-a2ff-4d9a-8a9c-af32777c568d admin admin] The token cache is being invalidate because project 65fba8ff6b34466b8c456924d9cae36f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:08:23.683586 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-2b0a2042-a2ff-4d9a-8a9c-af32777c568d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:23.686361 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-2b0a2042-a2ff-4d9a-8a9c-af32777c568d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:23.689151 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 822/1652] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:08:23 2022] DELETE /identity/v3/projects/65fba8ff6b34466b8c456924d9cae36f => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:08:23.694612 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83618555-5616-4624-b751-c51994b7eef1 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:23.695051 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83618555-5616-4624-b751-c51994b7eef1 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:23.695467 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83618555-5616-4624-b751-c51994b7eef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:23.717180 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 831/1653] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:08:23 2022] DELETE /identity/v3/users/af8181cc5e16451ba94e9f4c9c7ca629 => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:23.781241 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-83618555-5616-4624-b751-c51994b7eef1 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:23.784135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-83618555-5616-4624-b751-c51994b7eef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:23.919843 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 823/1654] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:08:23.928851 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1553029-b2ba-46bb-8586-7004bc7a1e76 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:23.929693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1553029-b2ba-46bb-8586-7004bc7a1e76 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:23.930360 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1553029-b2ba-46bb-8586-7004bc7a1e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:23.973140 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a1553029-b2ba-46bb-8586-7004bc7a1e76 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:23.975343 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a1553029-b2ba-46bb-8586-7004bc7a1e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:23.990170 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 832/1655] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:23.995586 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83618555-5616-4624-b751-c51994b7eef1 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:24.014512 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83618555-5616-4624-b751-c51994b7eef1 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:24.023575 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76db4f-5128-4fae-9f7f-4083d8d1b40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:24.024592 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76db4f-5128-4fae-9f7f-4083d8d1b40b admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:24.025018 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76db4f-5128-4fae-9f7f-4083d8d1b40b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:24.025510 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76db4f-5128-4fae-9f7f-4083d8d1b40b admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:24.026731 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec76db4f-5128-4fae-9f7f-4083d8d1b40b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:24.027691 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec76db4f-5128-4fae-9f7f-4083d8d1b40b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:24.030642 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec76db4f-5128-4fae-9f7f-4083d8d1b40b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:24.058917 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 824/1656] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:24.064967 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1553029-b2ba-46bb-8586-7004bc7a1e76 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:24.079821 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1553029-b2ba-46bb-8586-7004bc7a1e76 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:24.087314 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf4cef-3316-40a0-b187-0a5a5c249542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:24.088556 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf4cef-3316-40a0-b187-0a5a5c249542 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:24.089068 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf4cef-3316-40a0-b187-0a5a5c249542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:24.089576 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf4cef-3316-40a0-b187-0a5a5c249542 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:24.091250 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cf4cef-3316-40a0-b187-0a5a5c249542 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:24.095821 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cf4cef-3316-40a0-b187-0a5a5c249542 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:24.139750 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 833/1657] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:24 2022] POST /identity/v3/projects => generated 389 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:24.146736 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76db4f-5128-4fae-9f7f-4083d8d1b40b admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:24.168112 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76db4f-5128-4fae-9f7f-4083d8d1b40b admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:24.311087 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d703b2a-2782-4532-abf7-401c116737a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:24.311799 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d703b2a-2782-4532-abf7-401c116737a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:24.312087 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d703b2a-2782-4532-abf7-401c116737a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:24.312403 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d703b2a-2782-4532-abf7-401c116737a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:24.313471 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d703b2a-2782-4532-abf7-401c116737a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:24.324552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d703b2a-2782-4532-abf7-401c116737a0 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:24.411497 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 825/1658] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:24 2022] POST /identity/v3/users => generated 295 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:24.413151 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf4cef-3316-40a0-b187-0a5a5c249542 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:24.432033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf4cef-3316-40a0-b187-0a5a5c249542 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:24.434330 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d703b2a-2782-4532-abf7-401c116737a0 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:24.437226 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-4d703b2a-2782-4532-abf7-401c116737a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:24.438575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb30e3-bd43-4eb4-8fe0-20ef810bd5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:24.439233 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb30e3-bd43-4eb4-8fe0-20ef810bd5f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:24.439566 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb30e3-bd43-4eb4-8fe0-20ef810bd5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:24.442977 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb30e3-bd43-4eb4-8fe0-20ef810bd5f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:24.443887 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cb30e3-bd43-4eb4-8fe0-20ef810bd5f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:24.446113 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cb30e3-bd43-4eb4-8fe0-20ef810bd5f5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:24.460094 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d703b2a-2782-4532-abf7-401c116737a0 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:24.465945 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 834/1659] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:24.468001 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c2b8a-e46f-403d-8fb3-26c3ad981a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:24.469010 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c2b8a-e46f-403d-8fb3-26c3ad981a1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:24.469434 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c2b8a-e46f-403d-8fb3-26c3ad981a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:24.469843 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c2b8a-e46f-403d-8fb3-26c3ad981a1b admin admin] PATH_INFO: `/v3/projects/6ce580178de04321a708a358cf4af8a5` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:24.470097 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb30e3-bd43-4eb4-8fe0-20ef810bd5f5 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:24.471656 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0c2b8a-e46f-403d-8fb3-26c3ad981a1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ce580178de04321a708a358cf4af8a5)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:24.475928 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0c2b8a-e46f-403d-8fb3-26c3ad981a1b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:24.488486 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb30e3-bd43-4eb4-8fe0-20ef810bd5f5 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:24.497618 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7082d7-305f-43b6-8272-1b655a08f4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:24.499838 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7082d7-305f-43b6-8272-1b655a08f4fb admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:24.500222 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7082d7-305f-43b6-8272-1b655a08f4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:24.500668 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7082d7-305f-43b6-8272-1b655a08f4fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:24.501907 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7082d7-305f-43b6-8272-1b655a08f4fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:24.505387 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7082d7-305f-43b6-8272-1b655a08f4fb admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:24.521887 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 835/1660] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:24.526003 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7082d7-305f-43b6-8272-1b655a08f4fb admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:24.544868 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7082d7-305f-43b6-8272-1b655a08f4fb admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:24.555196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-9f0c2b8a-e46f-403d-8fb3-26c3ad981a1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ce580178de04321a708a358cf4af8a5', 'request_id': 'req-9f0c2b8a-e46f-403d-8fb3-26c3ad981a1b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:24.565281 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-9f0c2b8a-e46f-403d-8fb3-26c3ad981a1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ce580178de04321a708a358cf4af8a5', 'request_id': 'req-9f0c2b8a-e46f-403d-8fb3-26c3ad981a1b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:24.576629 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-9f0c2b8a-e46f-403d-8fb3-26c3ad981a1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ce580178de04321a708a358cf4af8a5', 'request_id': 'req-9f0c2b8a-e46f-403d-8fb3-26c3ad981a1b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:24.579792 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:24.593655 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-9f0c2b8a-e46f-403d-8fb3-26c3ad981a1b admin admin] The token cache is being invalidate because project 6ce580178de04321a708a358cf4af8a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:08:24.594178 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-9f0c2b8a-e46f-403d-8fb3-26c3ad981a1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:24.595447 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-9f0c2b8a-e46f-403d-8fb3-26c3ad981a1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:24.601506 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 826/1661] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:08:24 2022] DELETE /identity/v3/projects/6ce580178de04321a708a358cf4af8a5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:24.603201 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5887e-2685-415f-b0e9-5509a70b517d None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:24.603483 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5887e-2685-415f-b0e9-5509a70b517d None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:24.603934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5887e-2685-415f-b0e9-5509a70b517d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:24.621193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:24.639458 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-2ec5887e-2685-415f-b0e9-5509a70b517d None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:24.642298 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-2ec5887e-2685-415f-b0e9-5509a70b517d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:24.677157 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c8c01-3515-40bf-a102-837566ba03cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:24.678348 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c8c01-3515-40bf-a102-837566ba03cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:24.678631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c8c01-3515-40bf-a102-837566ba03cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:24.678901 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c8c01-3515-40bf-a102-837566ba03cb admin admin] PATH_INFO: `/v3/projects/c5cf41e589de4c7dacf4706a9c85407f/users/4e6b2e76c24c496993239e35e2b28f0f/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:24.702573 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750c8c01-3515-40bf-a102-837566ba03cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5cf41e589de4c7dacf4706a9c85407f, user_id=4e6b2e76c24c496993239e35e2b28f0f, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:24.709691 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750c8c01-3515-40bf-a102-837566ba03cb admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:24.757004 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 836/1662] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:24 2022] PUT /identity/v3/projects/c5cf41e589de4c7dacf4706a9c85407f/users/4e6b2e76c24c496993239e35e2b28f0f/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:24.759396 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11cd85-01ed-423c-bf7b-9bc73b3ce7a0 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:24.759684 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11cd85-01ed-423c-bf7b-9bc73b3ce7a0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:24.759955 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11cd85-01ed-423c-bf7b-9bc73b3ce7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:24.769084 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 827/1663] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:24.780361 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f27ba6-143e-440c-a30b-f4b771c57889 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:24.783641 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f27ba6-143e-440c-a30b-f4b771c57889 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:24.786624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f27ba6-143e-440c-a30b-f4b771c57889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:24.834342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-7e11cd85-01ed-423c-bf7b-9bc73b3ce7a0 None None] MFA Rules not processed for user `4e6b2e76c24c496993239e35e2b28f0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:24.835361 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-26f27ba6-143e-440c-a30b-f4b771c57889 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:24.836831 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-26f27ba6-143e-440c-a30b-f4b771c57889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:24.838372 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7e11cd85-01ed-423c-bf7b-9bc73b3ce7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:25.082396 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 828/1664] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:25.083747 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f27ba6-143e-440c-a30b-f4b771c57889 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:25.124454 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f27ba6-143e-440c-a30b-f4b771c57889 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:25.159104 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f893c-0e23-4944-9dfd-ea7a88d07715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:25.160631 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f893c-0e23-4944-9dfd-ea7a88d07715 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:25.161133 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f893c-0e23-4944-9dfd-ea7a88d07715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:25.161630 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f893c-0e23-4944-9dfd-ea7a88d07715 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:25.162179 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 837/1665] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:25.163229 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910f893c-0e23-4944-9dfd-ea7a88d07715 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:25.163784 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910f893c-0e23-4944-9dfd-ea7a88d07715 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:25.167156 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910f893c-0e23-4944-9dfd-ea7a88d07715 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:25.169361 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11cd85-01ed-423c-bf7b-9bc73b3ce7a0 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:25.173964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7e11cd85-01ed-423c-bf7b-9bc73b3ce7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:25.219400 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11cd85-01ed-423c-bf7b-9bc73b3ce7a0 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:25.227148 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 829/1666] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:25.229148 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f893c-0e23-4944-9dfd-ea7a88d07715 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:25.231915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383448f4-05d6-4f88-afc3-d67781b80d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:25.233973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383448f4-05d6-4f88-afc3-d67781b80d65 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:25.234534 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383448f4-05d6-4f88-afc3-d67781b80d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:25.234777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383448f4-05d6-4f88-afc3-d67781b80d65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:25.235551 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383448f4-05d6-4f88-afc3-d67781b80d65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:25.242011 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383448f4-05d6-4f88-afc3-d67781b80d65 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:25.257387 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f893c-0e23-4944-9dfd-ea7a88d07715 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:25.268016 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8bb072-772c-4532-b9cb-c910e6e79704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:25.269121 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8bb072-772c-4532-b9cb-c910e6e79704 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:25.269597 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8bb072-772c-4532-b9cb-c910e6e79704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:25.269952 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8bb072-772c-4532-b9cb-c910e6e79704 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:25.277561 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8bb072-772c-4532-b9cb-c910e6e79704 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:25.283973 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8bb072-772c-4532-b9cb-c910e6e79704 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:25.298309 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 838/1667] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:25 2022] POST /identity/v3/projects => generated 387 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:25.299413 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383448f4-05d6-4f88-afc3-d67781b80d65 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:25.381482 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 830/1668] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:25 2022] POST /identity/v3/projects => generated 369 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:25.382946 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8bb072-772c-4532-b9cb-c910e6e79704 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:25.424881 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383448f4-05d6-4f88-afc3-d67781b80d65 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:25.480645 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:25.488094 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8bb072-772c-4532-b9cb-c910e6e79704 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:25.497705 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405619b-7b0a-4161-ba3b-357ebe83d555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:25.498545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405619b-7b0a-4161-ba3b-357ebe83d555 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:25.499024 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405619b-7b0a-4161-ba3b-357ebe83d555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:25.499442 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405619b-7b0a-4161-ba3b-357ebe83d555 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:25.501057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405619b-7b0a-4161-ba3b-357ebe83d555 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:25.504254 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405619b-7b0a-4161-ba3b-357ebe83d555 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:25.531837 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:25.557057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 831/1669] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:25 2022] POST /identity/v3/users => generated 285 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:08:25.559150 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405619b-7b0a-4161-ba3b-357ebe83d555 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:25.576078 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405619b-7b0a-4161-ba3b-357ebe83d555 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:25.582484 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb2af6f-d98f-4160-8677-c045f92af861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:25.583380 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2af6f-d98f-4160-8677-c045f92af861 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:25.583803 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2af6f-d98f-4160-8677-c045f92af861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:25.584249 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2af6f-d98f-4160-8677-c045f92af861 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:25.585151 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb2af6f-d98f-4160-8677-c045f92af861 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:25.587424 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb2af6f-d98f-4160-8677-c045f92af861 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:25.593105 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34353f4-cf8b-4287-bdb6-67f348d0e07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:25.594670 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34353f4-cf8b-4287-bdb6-67f348d0e07b admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:25.594973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34353f4-cf8b-4287-bdb6-67f348d0e07b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:25.595756 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34353f4-cf8b-4287-bdb6-67f348d0e07b admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:25.597453 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34353f4-cf8b-4287-bdb6-67f348d0e07b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:25.602119 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34353f4-cf8b-4287-bdb6-67f348d0e07b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:25.607660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 832/1670] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:25.609596 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb2af6f-d98f-4160-8677-c045f92af861 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:25.621931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb2af6f-d98f-4160-8677-c045f92af861 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:25.628603 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b0510-e993-4d8e-a9dd-e77f1edd5f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:25.630103 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b0510-e993-4d8e-a9dd-e77f1edd5f55 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:25.630672 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b0510-e993-4d8e-a9dd-e77f1edd5f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:25.631149 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b0510-e993-4d8e-a9dd-e77f1edd5f55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:25.632153 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8b0510-e993-4d8e-a9dd-e77f1edd5f55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:25.633983 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8b0510-e993-4d8e-a9dd-e77f1edd5f55 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:25.658452 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 833/1671] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:25.660767 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b0510-e993-4d8e-a9dd-e77f1edd5f55 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:25.662073 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 839/1672] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:25 2022] POST /identity/v3/users => generated 300 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:25.667413 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34353f4-cf8b-4287-bdb6-67f348d0e07b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:25.682510 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b0510-e993-4d8e-a9dd-e77f1edd5f55 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:25.685005 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34353f4-cf8b-4287-bdb6-67f348d0e07b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:25.686850 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d19ad-8909-4b22-b79a-cab7e144c1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:25.687694 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d19ad-8909-4b22-b79a-cab7e144c1ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:25.687945 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d19ad-8909-4b22-b79a-cab7e144c1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:25.688170 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d19ad-8909-4b22-b79a-cab7e144c1ef admin admin] PATH_INFO: `/v3/projects/b42795ad0ec9407197883baa06958c59/users/34fc070a8a7841db8bc221a427179d1c/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:25.693817 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c7cb6-ebb9-4fc9-9abf-3b3c8382591a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:25.694746 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c7cb6-ebb9-4fc9-9abf-3b3c8382591a admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:25.695192 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c7cb6-ebb9-4fc9-9abf-3b3c8382591a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:25.695641 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c7cb6-ebb9-4fc9-9abf-3b3c8382591a admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:25.696533 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33c7cb6-ebb9-4fc9-9abf-3b3c8382591a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:25.698856 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33c7cb6-ebb9-4fc9-9abf-3b3c8382591a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:25.707898 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762d19ad-8909-4b22-b79a-cab7e144c1ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=b42795ad0ec9407197883baa06958c59, user_id=34fc070a8a7841db8bc221a427179d1c, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:25.711081 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762d19ad-8909-4b22-b79a-cab7e144c1ef admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:25.720414 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 840/1673] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:25.726965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c7cb6-ebb9-4fc9-9abf-3b3c8382591a admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:25.735810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 834/1674] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:25 2022] PUT /identity/v3/projects/b42795ad0ec9407197883baa06958c59/users/34fc070a8a7841db8bc221a427179d1c/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:25.743504 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c7cb6-ebb9-4fc9-9abf-3b3c8382591a admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:25.747054 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7880ecc-cb17-47ca-97a8-7dfc3f58c0f4 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:25.747360 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7880ecc-cb17-47ca-97a8-7dfc3f58c0f4 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:25.747630 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7880ecc-cb17-47ca-97a8-7dfc3f58c0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:25.786077 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-a7880ecc-cb17-47ca-97a8-7dfc3f58c0f4 None None] MFA Rules not processed for user `34fc070a8a7841db8bc221a427179d1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:25.788364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a7880ecc-cb17-47ca-97a8-7dfc3f58c0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:25.864710 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc20dbe-dd21-4de7-8d15-5877ea559242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:25.866677 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc20dbe-dd21-4de7-8d15-5877ea559242 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:25.867708 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc20dbe-dd21-4de7-8d15-5877ea559242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:25.868458 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc20dbe-dd21-4de7-8d15-5877ea559242 admin admin] PATH_INFO: `/v3/projects/36a6f758bcd0499cbb92ada1be9c8ae6/users/e078e6fe9668454099cd3d751a710155/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:25.893928 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 835/1675] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:25 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:08:25.895147 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7880ecc-cb17-47ca-97a8-7dfc3f58c0f4 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:25.912311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7880ecc-cb17-47ca-97a8-7dfc3f58c0f4 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:25.916449 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc20dbe-dd21-4de7-8d15-5877ea559242 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36a6f758bcd0499cbb92ada1be9c8ae6, user_id=e078e6fe9668454099cd3d751a710155, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:25.918945 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0673abb-64b7-4d3c-9536-364b7f6ffe32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:25.919906 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0673abb-64b7-4d3c-9536-364b7f6ffe32 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:25.920320 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0673abb-64b7-4d3c-9536-364b7f6ffe32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:25.920835 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0673abb-64b7-4d3c-9536-364b7f6ffe32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:25.921496 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc20dbe-dd21-4de7-8d15-5877ea559242 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:25.922039 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0673abb-64b7-4d3c-9536-364b7f6ffe32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:25.925354 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0673abb-64b7-4d3c-9536-364b7f6ffe32 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:25.968707 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 841/1676] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:25 2022] PUT /identity/v3/projects/36a6f758bcd0499cbb92ada1be9c8ae6/users/e078e6fe9668454099cd3d751a710155/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:25.974562 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 836/1677] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:25 2022] POST /identity/v3/projects => generated 367 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:25.974784 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc20dbe-dd21-4de7-8d15-5877ea559242 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:25.979705 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0673abb-64b7-4d3c-9536-364b7f6ffe32 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:26.127363 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0673abb-64b7-4d3c-9536-364b7f6ffe32 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:26.136966 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca38f8-7dfa-4373-9a80-a75a22e3dfcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:26.138946 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca38f8-7dfa-4373-9a80-a75a22e3dfcb admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:26.140739 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca38f8-7dfa-4373-9a80-a75a22e3dfcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:26.141821 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca38f8-7dfa-4373-9a80-a75a22e3dfcb admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:26.143098 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca38f8-7dfa-4373-9a80-a75a22e3dfcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:26.148827 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca38f8-7dfa-4373-9a80-a75a22e3dfcb admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:26.177029 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:26.185017 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc20dbe-dd21-4de7-8d15-5877ea559242 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:26.195463 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b4e59-e98c-4cc2-8063-37d73fd40b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:26.195463 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b4e59-e98c-4cc2-8063-37d73fd40b9d admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:26.195463 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b4e59-e98c-4cc2-8063-37d73fd40b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:26.195463 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b4e59-e98c-4cc2-8063-37d73fd40b9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:26.196180 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b4e59-e98c-4cc2-8063-37d73fd40b9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:26.197622 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b4e59-e98c-4cc2-8063-37d73fd40b9d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:26.205118 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 837/1678] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:25 2022] POST /identity/v3/users => generated 290 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:26.206271 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca38f8-7dfa-4373-9a80-a75a22e3dfcb admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:26.217704 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca38f8-7dfa-4373-9a80-a75a22e3dfcb admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:26.228617 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2060758-f007-4c3d-a01f-0caeab68174b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:26.229379 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2060758-f007-4c3d-a01f-0caeab68174b admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:26.230366 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2060758-f007-4c3d-a01f-0caeab68174b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:26.230902 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2060758-f007-4c3d-a01f-0caeab68174b admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:26.231737 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2060758-f007-4c3d-a01f-0caeab68174b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:26.233705 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2060758-f007-4c3d-a01f-0caeab68174b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:26.233864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 842/1679] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:26.241543 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b4e59-e98c-4cc2-8063-37d73fd40b9d admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:26.250623 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 838/1680] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:26.252699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2060758-f007-4c3d-a01f-0caeab68174b admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:26.284485 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b4e59-e98c-4cc2-8063-37d73fd40b9d admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:26.291646 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2060758-f007-4c3d-a01f-0caeab68174b admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:26.295597 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e6dec-6329-4995-a303-fd4a1c32647f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:26.298879 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e6dec-6329-4995-a303-fd4a1c32647f admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:26.299176 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e6dec-6329-4995-a303-fd4a1c32647f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:26.299493 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e6dec-6329-4995-a303-fd4a1c32647f admin admin] PATH_INFO: `/v3/projects/36a6f758bcd0499cbb92ada1be9c8ae6/users/e078e6fe9668454099cd3d751a710155/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:26.303433 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667e6dec-6329-4995-a303-fd4a1c32647f admin admin] RBAC: Authorizing `identity:create_grant(project_id=36a6f758bcd0499cbb92ada1be9c8ae6, user_id=e078e6fe9668454099cd3d751a710155, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:26.306770 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667e6dec-6329-4995-a303-fd4a1c32647f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:26.318753 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a253d-5c9b-462a-851e-7aab7cbe85e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:26.319708 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a253d-5c9b-462a-851e-7aab7cbe85e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:26.320098 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a253d-5c9b-462a-851e-7aab7cbe85e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:26.320502 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a253d-5c9b-462a-851e-7aab7cbe85e9 admin admin] PATH_INFO: `/v3/projects/1aeb8523bfb541c8aca84fbe2954da03/users/2e07e0829593453a81baa190e0c38b39/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:26.328993 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 843/1681] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:26 2022] PUT /identity/v3/projects/36a6f758bcd0499cbb92ada1be9c8ae6/users/e078e6fe9668454099cd3d751a710155/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:26.330541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98ed39-5b54-46fc-a87e-b0584aa42171 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:26.330860 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98ed39-5b54-46fc-a87e-b0584aa42171 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:26.331161 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98ed39-5b54-46fc-a87e-b0584aa42171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:26.373439 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214a253d-5c9b-462a-851e-7aab7cbe85e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aeb8523bfb541c8aca84fbe2954da03, user_id=2e07e0829593453a81baa190e0c38b39, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:26.376669 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214a253d-5c9b-462a-851e-7aab7cbe85e9 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:26.386851 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-9a98ed39-5b54-46fc-a87e-b0584aa42171 None None] MFA Rules not processed for user `87f35e960544403ea30ba36d878a86cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:26.387897 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9a98ed39-5b54-46fc-a87e-b0584aa42171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:26.402552 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:26.417620 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 839/1682] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:26 2022] PUT /identity/v3/projects/1aeb8523bfb541c8aca84fbe2954da03/users/2e07e0829593453a81baa190e0c38b39/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:26.422185 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd2032-0557-4eb2-b523-e1b3d4e65417 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:26.422545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd2032-0557-4eb2-b523-e1b3d4e65417 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:26.422889 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd2032-0557-4eb2-b523-e1b3d4e65417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:26.497364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-d3dd2032-0557-4eb2-b523-e1b3d4e65417 None None] MFA Rules not processed for user `e078e6fe9668454099cd3d751a710155`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:26.503975 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d3dd2032-0557-4eb2-b523-e1b3d4e65417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:26.525447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 844/1683] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:26.525447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98ed39-5b54-46fc-a87e-b0584aa42171 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:26.719226 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98ed39-5b54-46fc-a87e-b0584aa42171 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:26.750939 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 840/1684] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:26 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:26.753165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd2032-0557-4eb2-b523-e1b3d4e65417 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:26.760963 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d3dd2032-0557-4eb2-b523-e1b3d4e65417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:26.852409 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4880d-3052-479f-ad15-79a2fff7d3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:26.853747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4880d-3052-479f-ad15-79a2fff7d3d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:26.854212 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4880d-3052-479f-ad15-79a2fff7d3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:26.854684 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4880d-3052-479f-ad15-79a2fff7d3d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:26.856075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d4880d-3052-479f-ad15-79a2fff7d3d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:26.863461 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d4880d-3052-479f-ad15-79a2fff7d3d4 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:26.904714 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 845/1685] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:26.905722 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4880d-3052-479f-ad15-79a2fff7d3d4 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:26.909841 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f1d4880d-3052-479f-ad15-79a2fff7d3d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:26.916931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd2032-0557-4eb2-b523-e1b3d4e65417 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:26.923862 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e72c3c-c6e2-4113-88bf-c9a0ca8bfbbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:26.924793 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e72c3c-c6e2-4113-88bf-c9a0ca8bfbbf service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:26.925179 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e72c3c-c6e2-4113-88bf-c9a0ca8bfbbf service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:26.925736 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e72c3c-c6e2-4113-88bf-c9a0ca8bfbbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:27.018993 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:27.042059 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e72c3c-c6e2-4113-88bf-c9a0ca8bfbbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:27.043770 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e72c3c-c6e2-4113-88bf-c9a0ca8bfbbf service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:27.074871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 841/1686] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:27.076204 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e72c3c-c6e2-4113-88bf-c9a0ca8bfbbf service nova] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:27.098270 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4880d-3052-479f-ad15-79a2fff7d3d4 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:27.120351 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47f303-a16f-4236-9d05-0282d00c22cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:27.121479 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47f303-a16f-4236-9d05-0282d00c22cb service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:27.121833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e72c3c-c6e2-4113-88bf-c9a0ca8bfbbf service nova] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:27.121980 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47f303-a16f-4236-9d05-0282d00c22cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:27.122469 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47f303-a16f-4236-9d05-0282d00c22cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:27.142754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64db28-6c5a-4d01-9774-a3ce6bc7301a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:27.144626 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64db28-6c5a-4d01-9774-a3ce6bc7301a admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:27.145494 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64db28-6c5a-4d01-9774-a3ce6bc7301a admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:27.147374 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64db28-6c5a-4d01-9774-a3ce6bc7301a admin admin] PATH_INFO: `/v3/projects/1aeb8523bfb541c8aca84fbe2954da03/users/2e07e0829593453a81baa190e0c38b39/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:27.154049 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c64db28-6c5a-4d01-9774-a3ce6bc7301a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aeb8523bfb541c8aca84fbe2954da03, user_id=2e07e0829593453a81baa190e0c38b39, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:27.166107 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c64db28-6c5a-4d01-9774-a3ce6bc7301a admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:27.182229 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e47f303-a16f-4236-9d05-0282d00c22cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:27.186497 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e47f303-a16f-4236-9d05-0282d00c22cb service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:27.197534 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:27.227972 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 842/1687] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:27 2022] PUT /identity/v3/projects/1aeb8523bfb541c8aca84fbe2954da03/users/2e07e0829593453a81baa190e0c38b39/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:27.248112 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960b3d5-c660-4ecc-9f2a-5f6db7f1eddb None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:27.248937 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960b3d5-c660-4ecc-9f2a-5f6db7f1eddb None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:27.249644 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960b3d5-c660-4ecc-9f2a-5f6db7f1eddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:27.250406 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 846/1688] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:26 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:27.309609 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47f303-a16f-4236-9d05-0282d00c22cb service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:27.312771 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2e47f303-a16f-4236-9d05-0282d00c22cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:27.403514 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-e960b3d5-c660-4ecc-9f2a-5f6db7f1eddb None None] MFA Rules not processed for user `2e07e0829593453a81baa190e0c38b39`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:27.405440 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47f303-a16f-4236-9d05-0282d00c22cb service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:27.416549 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e960b3d5-c660-4ecc-9f2a-5f6db7f1eddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:27.685753 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dfbf70-5e3e-4984-a1b6-763f64feecc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:27.686912 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dfbf70-5e3e-4984-a1b6-763f64feecc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:27.687410 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dfbf70-5e3e-4984-a1b6-763f64feecc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:27.687904 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dfbf70-5e3e-4984-a1b6-763f64feecc8 admin admin] PATH_INFO: `/v3/users/83360ac42cdc4dbabb3016177e523803` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:27.695455 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dfbf70-5e3e-4984-a1b6-763f64feecc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83360ac42cdc4dbabb3016177e523803)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:27.699321 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dfbf70-5e3e-4984-a1b6-763f64feecc8 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:27.909086 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 843/1689] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:27 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 661 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:27.938147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-08dfbf70-5e3e-4984-a1b6-763f64feecc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83360ac42cdc4dbabb3016177e523803'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:27.939193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-08dfbf70-5e3e-4984-a1b6-763f64feecc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83360ac42cdc4dbabb3016177e523803'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:27.957060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-08dfbf70-5e3e-4984-a1b6-763f64feecc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83360ac42cdc4dbabb3016177e523803'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:27.968863 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-08dfbf70-5e3e-4984-a1b6-763f64feecc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83360ac42cdc4dbabb3016177e523803'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:27.969844 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-08dfbf70-5e3e-4984-a1b6-763f64feecc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83360ac42cdc4dbabb3016177e523803'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:27.993304 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 847/1690] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:08:27 2022] DELETE /identity/v3/users/83360ac42cdc4dbabb3016177e523803 => generated 0 bytes in 681 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:08:27.996715 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3b631-5bb2-4a05-98c7-a0df44426d10 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:27.997432 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3b631-5bb2-4a05-98c7-a0df44426d10 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:27.998197 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3b631-5bb2-4a05-98c7-a0df44426d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:28.064930 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dfbf70-5e3e-4984-a1b6-763f64feecc8 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:28.067475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-08dfbf70-5e3e-4984-a1b6-763f64feecc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:28.117606 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-57f3b631-5bb2-4a05-98c7-a0df44426d10 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:28.120342 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-57f3b631-5bb2-4a05-98c7-a0df44426d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:28.149011 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dfbf70-5e3e-4984-a1b6-763f64feecc8 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:28.154514 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29e7be-91a1-422c-848b-9dbbd00e564b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:28.155204 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29e7be-91a1-422c-848b-9dbbd00e564b service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:28.155488 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29e7be-91a1-422c-848b-9dbbd00e564b service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:28.155760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29e7be-91a1-422c-848b-9dbbd00e564b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:28.158330 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0d29e7be-91a1-422c-848b-9dbbd00e564b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:28.219932 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:28.280904 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29e7be-91a1-422c-848b-9dbbd00e564b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:28.285462 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29e7be-91a1-422c-848b-9dbbd00e564b service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:28.319986 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 848/1691] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:28 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:28.455398 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 844/1692] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:28.550562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29e7be-91a1-422c-848b-9dbbd00e564b service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:28.552363 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0d29e7be-91a1-422c-848b-9dbbd00e564b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:28.574775 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3b631-5bb2-4a05-98c7-a0df44426d10 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:28.578631 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-57f3b631-5bb2-4a05-98c7-a0df44426d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:28.647431 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3b631-5bb2-4a05-98c7-a0df44426d10 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:28.674381 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944b36ec-0d56-4943-81a9-2e2f63ea405b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:28.675321 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b36ec-0d56-4943-81a9-2e2f63ea405b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:28.676749 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b36ec-0d56-4943-81a9-2e2f63ea405b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:28.677034 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b36ec-0d56-4943-81a9-2e2f63ea405b admin admin] PATH_INFO: `/v3/users/4e6b2e76c24c496993239e35e2b28f0f` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:28.680704 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944b36ec-0d56-4943-81a9-2e2f63ea405b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e6b2e76c24c496993239e35e2b28f0f)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:28.684343 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944b36ec-0d56-4943-81a9-2e2f63ea405b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:28.712713 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29e7be-91a1-422c-848b-9dbbd00e564b service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:28.717322 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78d2cc-7e43-4e03-a127-3c141d1e1a15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:28.718002 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78d2cc-7e43-4e03-a127-3c141d1e1a15 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:28.718289 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78d2cc-7e43-4e03-a127-3c141d1e1a15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:28.718547 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78d2cc-7e43-4e03-a127-3c141d1e1a15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:28.735241 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd78d2cc-7e43-4e03-a127-3c141d1e1a15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:28.736941 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd78d2cc-7e43-4e03-a127-3c141d1e1a15 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:28.770996 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 849/1693] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:28.772102 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78d2cc-7e43-4e03-a127-3c141d1e1a15 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:28.775278 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fd78d2cc-7e43-4e03-a127-3c141d1e1a15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:28.808497 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-944b36ec-0d56-4943-81a9-2e2f63ea405b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e6b2e76c24c496993239e35e2b28f0f'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:28.809342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78d2cc-7e43-4e03-a127-3c141d1e1a15 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:28.810754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-944b36ec-0d56-4943-81a9-2e2f63ea405b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e6b2e76c24c496993239e35e2b28f0f'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:28.818199 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18d53b-3dc1-4659-8e17-8f440e7f25f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:28.818453 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:28.821975 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18d53b-3dc1-4659-8e17-8f440e7f25f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:28.822468 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18d53b-3dc1-4659-8e17-8f440e7f25f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:28.823102 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18d53b-3dc1-4659-8e17-8f440e7f25f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:28.828370 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4c18d53b-3dc1-4659-8e17-8f440e7f25f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:28.833516 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-944b36ec-0d56-4943-81a9-2e2f63ea405b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e6b2e76c24c496993239e35e2b28f0f'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:28.850022 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-944b36ec-0d56-4943-81a9-2e2f63ea405b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e6b2e76c24c496993239e35e2b28f0f'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:28.884176 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-944b36ec-0d56-4943-81a9-2e2f63ea405b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e6b2e76c24c496993239e35e2b28f0f'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:28.905216 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 845/1694] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:08:28 2022] DELETE /identity/v3/users/4e6b2e76c24c496993239e35e2b28f0f => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:28.910206 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01fdfa-8073-44a3-b674-c22c8e92eeb5 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:28.910948 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01fdfa-8073-44a3-b674-c22c8e92eeb5 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:28.911660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01fdfa-8073-44a3-b674-c22c8e92eeb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:28.991738 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-9b01fdfa-8073-44a3-b674-c22c8e92eeb5 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:28.998374 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-9b01fdfa-8073-44a3-b674-c22c8e92eeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:29.013578 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c18d53b-3dc1-4659-8e17-8f440e7f25f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:29.020932 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c18d53b-3dc1-4659-8e17-8f440e7f25f8 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:29.057676 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 850/1695] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:28 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:29.162924 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 846/1696] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:29.195320 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18d53b-3dc1-4659-8e17-8f440e7f25f8 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:29.197207 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4c18d53b-3dc1-4659-8e17-8f440e7f25f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:29.217482 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18d53b-3dc1-4659-8e17-8f440e7f25f8 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:29.376844 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bede20-0bf0-4a42-a0e8-65be617abca0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:29.377861 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bede20-0bf0-4a42-a0e8-65be617abca0 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:29.378252 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bede20-0bf0-4a42-a0e8-65be617abca0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:29.380764 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bede20-0bf0-4a42-a0e8-65be617abca0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:29.410202 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bede20-0bf0-4a42-a0e8-65be617abca0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:29.413315 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bede20-0bf0-4a42-a0e8-65be617abca0 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:29.447671 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 851/1697] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:30.864389 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a136e5-76b5-4ad7-a573-f0b0cdbcb3c1 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:30.865204 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a136e5-76b5-4ad7-a573-f0b0cdbcb3c1 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:30.865204 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a136e5-76b5-4ad7-a573-f0b0cdbcb3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:30.939757 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bede20-0bf0-4a42-a0e8-65be617abca0 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:30.943398 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f7bede20-0bf0-4a42-a0e8-65be617abca0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:30.965989 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-95a136e5-76b5-4ad7-a573-f0b0cdbcb3c1 None None] MFA Rules not processed for user `ed6e0b24487342c1a9246579c0854653`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:30.968093 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-95a136e5-76b5-4ad7-a573-f0b0cdbcb3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:30.990060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bede20-0bf0-4a42-a0e8-65be617abca0 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:31.003819 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670469c1-3cc6-45db-96f3-208d998f7b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:31.005180 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670469c1-3cc6-45db-96f3-208d998f7b81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:31.005867 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670469c1-3cc6-45db-96f3-208d998f7b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:31.006437 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670469c1-3cc6-45db-96f3-208d998f7b81 admin admin] PATH_INFO: `/v3/projects/c5cf41e589de4c7dacf4706a9c85407f` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:31.008499 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670469c1-3cc6-45db-96f3-208d998f7b81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5cf41e589de4c7dacf4706a9c85407f)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:31.011332 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670469c1-3cc6-45db-96f3-208d998f7b81 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:31.025213 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:31.127996 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-670469c1-3cc6-45db-96f3-208d998f7b81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5cf41e589de4c7dacf4706a9c85407f', 'request_id': 'req-670469c1-3cc6-45db-96f3-208d998f7b81'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:31.156713 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-670469c1-3cc6-45db-96f3-208d998f7b81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5cf41e589de4c7dacf4706a9c85407f', 'request_id': 'req-670469c1-3cc6-45db-96f3-208d998f7b81'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:31.174723 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-670469c1-3cc6-45db-96f3-208d998f7b81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5cf41e589de4c7dacf4706a9c85407f', 'request_id': 'req-670469c1-3cc6-45db-96f3-208d998f7b81'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:31.206837 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-670469c1-3cc6-45db-96f3-208d998f7b81 admin admin] The token cache is being invalidate because project c5cf41e589de4c7dacf4706a9c85407f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:08:31.207156 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-670469c1-3cc6-45db-96f3-208d998f7b81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:31.209517 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-670469c1-3cc6-45db-96f3-208d998f7b81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:31.213111 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 852/1698] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:08:30 2022] DELETE /identity/v3/projects/c5cf41e589de4c7dacf4706a9c85407f => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:08:31.217175 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670469c1-3cc6-45db-96f3-208d998f7b81 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:31.225220 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-670469c1-3cc6-45db-96f3-208d998f7b81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:31.264570 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 847/1699] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:31.282945 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b232d-3ad5-48b7-bbb1-fa4049d049b7 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:31.283442 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b232d-3ad5-48b7-bbb1-fa4049d049b7 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:31.283949 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b232d-3ad5-48b7-bbb1-fa4049d049b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:31.414716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-b64b232d-3ad5-48b7-bbb1-fa4049d049b7 None None] MFA Rules not processed for user `ed6e0b24487342c1a9246579c0854653`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:31.443994 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b64b232d-3ad5-48b7-bbb1-fa4049d049b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:31.495975 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670469c1-3cc6-45db-96f3-208d998f7b81 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:31.520467 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58311cbb-3df3-4c60-86ac-a4708e88426d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:31.524584 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58311cbb-3df3-4c60-86ac-a4708e88426d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:31.527689 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58311cbb-3df3-4c60-86ac-a4708e88426d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:31.544291 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58311cbb-3df3-4c60-86ac-a4708e88426d admin admin] PATH_INFO: `/v3/users/e078e6fe9668454099cd3d751a710155` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:31.551258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58311cbb-3df3-4c60-86ac-a4708e88426d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e078e6fe9668454099cd3d751a710155)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:31.555513 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58311cbb-3df3-4c60-86ac-a4708e88426d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:31.627335 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 848/1700] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:08:31.629303 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b232d-3ad5-48b7-bbb1-fa4049d049b7 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:31.638409 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b64b232d-3ad5-48b7-bbb1-fa4049d049b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:31.668921 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:31.689573 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b232d-3ad5-48b7-bbb1-fa4049d049b7 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:31.709937 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-58311cbb-3df3-4c60-86ac-a4708e88426d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e078e6fe9668454099cd3d751a710155'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:31.710772 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-58311cbb-3df3-4c60-86ac-a4708e88426d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e078e6fe9668454099cd3d751a710155'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:31.714734 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce89f34-51f7-4cdd-8bff-f72a616bcab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:31.721232 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce89f34-51f7-4cdd-8bff-f72a616bcab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:31.722861 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce89f34-51f7-4cdd-8bff-f72a616bcab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:31.723668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce89f34-51f7-4cdd-8bff-f72a616bcab9 admin admin] PATH_INFO: `/v3/projects/f209a9617cbe40f394023a7a1ba12b81` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:31.727716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce89f34-51f7-4cdd-8bff-f72a616bcab9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f209a9617cbe40f394023a7a1ba12b81)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:31.733306 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce89f34-51f7-4cdd-8bff-f72a616bcab9 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:31.739544 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-58311cbb-3df3-4c60-86ac-a4708e88426d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e078e6fe9668454099cd3d751a710155'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:31.749661 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-58311cbb-3df3-4c60-86ac-a4708e88426d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e078e6fe9668454099cd3d751a710155'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:31.750580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-58311cbb-3df3-4c60-86ac-a4708e88426d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e078e6fe9668454099cd3d751a710155'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:31.777914 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 853/1701] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:08:31 2022] DELETE /identity/v3/users/e078e6fe9668454099cd3d751a710155 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:31.777914 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58311cbb-3df3-4c60-86ac-a4708e88426d admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:31.793382 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-58311cbb-3df3-4c60-86ac-a4708e88426d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:31.829749 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:31.953277 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8ce89f34-51f7-4cdd-8bff-f72a616bcab9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f209a9617cbe40f394023a7a1ba12b81', 'request_id': 'req-8ce89f34-51f7-4cdd-8bff-f72a616bcab9'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:31.978582 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8ce89f34-51f7-4cdd-8bff-f72a616bcab9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f209a9617cbe40f394023a7a1ba12b81', 'request_id': 'req-8ce89f34-51f7-4cdd-8bff-f72a616bcab9'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:31.994957 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8ce89f34-51f7-4cdd-8bff-f72a616bcab9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f209a9617cbe40f394023a7a1ba12b81', 'request_id': 'req-8ce89f34-51f7-4cdd-8bff-f72a616bcab9'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:32.029368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58311cbb-3df3-4c60-86ac-a4708e88426d admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:32.033872 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8ce89f34-51f7-4cdd-8bff-f72a616bcab9 admin admin] The token cache is being invalidate because project f209a9617cbe40f394023a7a1ba12b81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:08:32.035223 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8ce89f34-51f7-4cdd-8bff-f72a616bcab9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:32.044689 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8ce89f34-51f7-4cdd-8bff-f72a616bcab9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:32.051916 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 849/1702] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:08:31 2022] DELETE /identity/v3/projects/f209a9617cbe40f394023a7a1ba12b81 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:32.058230 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a481f35-f83d-4597-bb27-a9819529f71a None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:32.059791 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a481f35-f83d-4597-bb27-a9819529f71a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:32.061193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a481f35-f83d-4597-bb27-a9819529f71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:32.115728 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-9a481f35-f83d-4597-bb27-a9819529f71a None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:32.118098 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-9a481f35-f83d-4597-bb27-a9819529f71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:32.301554 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 850/1703] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:32.311373 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2173516-ed81-4f56-afa0-548c9719b205 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:32.312141 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2173516-ed81-4f56-afa0-548c9719b205 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:32.312750 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2173516-ed81-4f56-afa0-548c9719b205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:32.341023 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4ee2fb-e225-4b96-83c4-705359a4137d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:32.355464 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ee2fb-e225-4b96-83c4-705359a4137d service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:32.360313 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ee2fb-e225-4b96-83c4-705359a4137d service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:32.360771 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ee2fb-e225-4b96-83c4-705359a4137d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:32.367792 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-b2173516-ed81-4f56-afa0-548c9719b205 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:32.369439 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-aa4ee2fb-e225-4b96-83c4-705359a4137d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:32.376244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b2173516-ed81-4f56-afa0-548c9719b205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:32.420176 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 851/1704] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:32.423401 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2173516-ed81-4f56-afa0-548c9719b205 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:32.441928 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2173516-ed81-4f56-afa0-548c9719b205 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:32.447756 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58cb0d-d7fb-4563-8c51-220f151ff43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:32.450294 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58cb0d-d7fb-4563-8c51-220f151ff43e admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:32.458934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58cb0d-d7fb-4563-8c51-220f151ff43e admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:32.462685 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58cb0d-d7fb-4563-8c51-220f151ff43e admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:32.463801 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa58cb0d-d7fb-4563-8c51-220f151ff43e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:32.464345 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa58cb0d-d7fb-4563-8c51-220f151ff43e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:32.470694 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa58cb0d-d7fb-4563-8c51-220f151ff43e admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:32.545179 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 852/1705] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:32.553500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58cb0d-d7fb-4563-8c51-220f151ff43e admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:32.584370 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4ee2fb-e225-4b96-83c4-705359a4137d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:32.591206 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58cb0d-d7fb-4563-8c51-220f151ff43e admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:32.592233 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4ee2fb-e225-4b96-83c4-705359a4137d service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:32.609338 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb89ac-ef9c-4960-b60d-4a9ee48b4980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:32.610489 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb89ac-ef9c-4960-b60d-4a9ee48b4980 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:32.611898 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb89ac-ef9c-4960-b60d-4a9ee48b4980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:32.612431 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb89ac-ef9c-4960-b60d-4a9ee48b4980 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:32.622372 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb89ac-ef9c-4960-b60d-4a9ee48b4980 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:32.626582 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 854/1706] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 853 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:32.627561 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb89ac-ef9c-4960-b60d-4a9ee48b4980 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:32.741954 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4ee2fb-e225-4b96-83c4-705359a4137d service nova] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:32.745359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-aa4ee2fb-e225-4b96-83c4-705359a4137d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:32.780871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 853/1707] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:32 2022] POST /identity/v3/projects => generated 367 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:32.786846 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb89ac-ef9c-4960-b60d-4a9ee48b4980 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:32.830330 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4ee2fb-e225-4b96-83c4-705359a4137d service nova] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:32.981911 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:33.021760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f542ffe2-32bb-4dc2-a3b6-d882410234d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:33.025253 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542ffe2-32bb-4dc2-a3b6-d882410234d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:33.025606 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542ffe2-32bb-4dc2-a3b6-d882410234d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:33.026930 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542ffe2-32bb-4dc2-a3b6-d882410234d3 admin admin] PATH_INFO: `/v3/projects/36a6f758bcd0499cbb92ada1be9c8ae6` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:33.027960 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:33.034554 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f542ffe2-32bb-4dc2-a3b6-d882410234d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36a6f758bcd0499cbb92ada1be9c8ae6)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:33.043789 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f542ffe2-32bb-4dc2-a3b6-d882410234d3 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:33.075287 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb89ac-ef9c-4960-b60d-4a9ee48b4980 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:33.087700 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b49bd-12ac-4def-a2f9-65ec2c0002bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:33.088703 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b49bd-12ac-4def-a2f9-65ec2c0002bf admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:33.090407 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b49bd-12ac-4def-a2f9-65ec2c0002bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:33.090656 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b49bd-12ac-4def-a2f9-65ec2c0002bf admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:33.092765 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113b49bd-12ac-4def-a2f9-65ec2c0002bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:33.100819 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113b49bd-12ac-4def-a2f9-65ec2c0002bf admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:33.155072 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 854/1708] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:32 2022] POST /identity/v3/users => generated 284 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:33.156024 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b49bd-12ac-4def-a2f9-65ec2c0002bf admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:33.161818 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-113b49bd-12ac-4def-a2f9-65ec2c0002bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:33.177052 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f542ffe2-32bb-4dc2-a3b6-d882410234d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36a6f758bcd0499cbb92ada1be9c8ae6', 'request_id': 'req-f542ffe2-32bb-4dc2-a3b6-d882410234d3'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:33.208207 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f542ffe2-32bb-4dc2-a3b6-d882410234d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36a6f758bcd0499cbb92ada1be9c8ae6', 'request_id': 'req-f542ffe2-32bb-4dc2-a3b6-d882410234d3'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:33.215923 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b49bd-12ac-4def-a2f9-65ec2c0002bf admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:33.222872 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f542ffe2-32bb-4dc2-a3b6-d882410234d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36a6f758bcd0499cbb92ada1be9c8ae6', 'request_id': 'req-f542ffe2-32bb-4dc2-a3b6-d882410234d3'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:33.256654 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f542ffe2-32bb-4dc2-a3b6-d882410234d3 admin admin] The token cache is being invalidate because project 36a6f758bcd0499cbb92ada1be9c8ae6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:08:33.257018 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f542ffe2-32bb-4dc2-a3b6-d882410234d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:33.258115 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f542ffe2-32bb-4dc2-a3b6-d882410234d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:33.264046 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 855/1709] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:08:32 2022] DELETE /identity/v3/projects/36a6f758bcd0499cbb92ada1be9c8ae6 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:33.266386 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f542ffe2-32bb-4dc2-a3b6-d882410234d3 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:33.276032 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f542ffe2-32bb-4dc2-a3b6-d882410234d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:33.445007 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d2b11-b49d-411b-907b-1ec60bf6b4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:33.445747 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d2b11-b49d-411b-907b-1ec60bf6b4a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:33.446192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d2b11-b49d-411b-907b-1ec60bf6b4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:33.446720 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d2b11-b49d-411b-907b-1ec60bf6b4a7 admin admin] PATH_INFO: `/v3/users/a7dd1ba60fb64c18ba5f199d8e27d167` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:33.449586 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d2b11-b49d-411b-907b-1ec60bf6b4a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7dd1ba60fb64c18ba5f199d8e27d167)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:33.451591 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d2b11-b49d-411b-907b-1ec60bf6b4a7 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:33.507889 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f542ffe2-32bb-4dc2-a3b6-d882410234d3 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:33.524466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ea5a7-aed5-46a7-a21c-77de65f145d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:33.525213 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ea5a7-aed5-46a7-a21c-77de65f145d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:33.525498 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ea5a7-aed5-46a7-a21c-77de65f145d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:33.525797 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ea5a7-aed5-46a7-a21c-77de65f145d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:33.526747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906ea5a7-aed5-46a7-a21c-77de65f145d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:33.530930 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906ea5a7-aed5-46a7-a21c-77de65f145d0 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:33.587936 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 856/1710] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:33.592070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b862fc49-eef1-4f20-ad69-bc5d6681d327 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:33.592374 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b862fc49-eef1-4f20-ad69-bc5d6681d327 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:33.592547 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b862fc49-eef1-4f20-ad69-bc5d6681d327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:33.610303 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-6a7d2b11-b49d-411b-907b-1ec60bf6b4a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7dd1ba60fb64c18ba5f199d8e27d167'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:33.611521 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-6a7d2b11-b49d-411b-907b-1ec60bf6b4a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7dd1ba60fb64c18ba5f199d8e27d167'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:33.626517 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-6a7d2b11-b49d-411b-907b-1ec60bf6b4a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7dd1ba60fb64c18ba5f199d8e27d167'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:33.638892 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b862fc49-eef1-4f20-ad69-bc5d6681d327 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:33.641664 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b862fc49-eef1-4f20-ad69-bc5d6681d327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:33.644296 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-6a7d2b11-b49d-411b-907b-1ec60bf6b4a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7dd1ba60fb64c18ba5f199d8e27d167'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:33.669509 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 857/1711] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:33.670146 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b862fc49-eef1-4f20-ad69-bc5d6681d327 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:33.675033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b862fc49-eef1-4f20-ad69-bc5d6681d327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:33.689082 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-6a7d2b11-b49d-411b-907b-1ec60bf6b4a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7dd1ba60fb64c18ba5f199d8e27d167'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:33.704252 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 855/1712] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:08:33 2022] DELETE /identity/v3/users/a7dd1ba60fb64c18ba5f199d8e27d167 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:33.707866 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489b017-b938-4139-9512-27b908415861 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:33.708713 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489b017-b938-4139-9512-27b908415861 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:33.708996 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489b017-b938-4139-9512-27b908415861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:33.714744 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b862fc49-eef1-4f20-ad69-bc5d6681d327 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:33.780615 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-0489b017-b938-4139-9512-27b908415861 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:33.783500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0489b017-b938-4139-9512-27b908415861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:33.859490 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94364a6b-ed46-4494-93b6-cf58e13eea77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:33.860570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94364a6b-ed46-4494-93b6-cf58e13eea77 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:33.861108 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94364a6b-ed46-4494-93b6-cf58e13eea77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:33.861508 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94364a6b-ed46-4494-93b6-cf58e13eea77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:33.862384 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94364a6b-ed46-4494-93b6-cf58e13eea77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:33.865875 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94364a6b-ed46-4494-93b6-cf58e13eea77 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:33.882613 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:33.894828 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 858/1713] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:33.898457 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaee968-f8ff-4d9c-8d26-00b88c672a1e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:33.899734 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaee968-f8ff-4d9c-8d26-00b88c672a1e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:33.900414 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaee968-f8ff-4d9c-8d26-00b88c672a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:33.915111 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 856/1714] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:33.916681 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265c338-7848-4fb6-a6c7-d50248dd5517 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:33.916966 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265c338-7848-4fb6-a6c7-d50248dd5517 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:33.917247 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265c338-7848-4fb6-a6c7-d50248dd5517 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:33.917369 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:33.920684 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 857/1715] 10.222.0.45 () {54 vars in 912 bytes} [Mon Aug 1 14:08:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:08:33.921701 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265c338-7848-4fb6-a6c7-d50248dd5517 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:33.956033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265c338-7848-4fb6-a6c7-d50248dd5517 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:33.960785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-cdaee968-f8ff-4d9c-8d26-00b88c672a1e None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:33.963183 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-cdaee968-f8ff-4d9c-8d26-00b88c672a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:33.976157 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652929af-de9d-4d4c-b70b-bf7e3fdbb608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:33.978896 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652929af-de9d-4d4c-b70b-bf7e3fdbb608 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:33.980605 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652929af-de9d-4d4c-b70b-bf7e3fdbb608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:33.981121 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652929af-de9d-4d4c-b70b-bf7e3fdbb608 admin admin] PATH_INFO: `/v3/projects/6c620caee24342328e7bd4d44cbec2de/users/3f61e3dd7eb1459381df05af41987bea/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:33.991573 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 859/1716] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:33.993622 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaee968-f8ff-4d9c-8d26-00b88c672a1e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:34.028786 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaee968-f8ff-4d9c-8d26-00b88c672a1e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:34.039373 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652929af-de9d-4d4c-b70b-bf7e3fdbb608 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c620caee24342328e7bd4d44cbec2de, user_id=3f61e3dd7eb1459381df05af41987bea, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:34.046784 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652929af-de9d-4d4c-b70b-bf7e3fdbb608 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:34.047081 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfdf498-3af5-4848-9253-d23ee3e3a3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:34.048317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfdf498-3af5-4848-9253-d23ee3e3a3ca admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:34.049525 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfdf498-3af5-4848-9253-d23ee3e3a3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:34.050200 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfdf498-3af5-4848-9253-d23ee3e3a3ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:34.051428 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfdf498-3af5-4848-9253-d23ee3e3a3ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:34.053731 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfdf498-3af5-4848-9253-d23ee3e3a3ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:34.057478 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfdf498-3af5-4848-9253-d23ee3e3a3ca admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:34.076307 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 858/1717] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:33 2022] PUT /identity/v3/projects/6c620caee24342328e7bd4d44cbec2de/users/3f61e3dd7eb1459381df05af41987bea/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:34.077727 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352d08a-d084-4692-8d8a-16e021acaefa None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:34.078014 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352d08a-d084-4692-8d8a-16e021acaefa None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:34.078283 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352d08a-d084-4692-8d8a-16e021acaefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:34.087868 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 860/1718] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:34.088885 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfdf498-3af5-4848-9253-d23ee3e3a3ca admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:34.100638 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8cfdf498-3af5-4848-9253-d23ee3e3a3ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:34.215620 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-1352d08a-d084-4692-8d8a-16e021acaefa None None] MFA Rules not processed for user `6dd44b7a78594288b6aabb393fd6324a`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:34.217245 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1352d08a-d084-4692-8d8a-16e021acaefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:34.251014 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfdf498-3af5-4848-9253-d23ee3e3a3ca admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:34.399255 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e27a7c-f387-47df-8bfd-13c940c8a85b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:34.402649 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e27a7c-f387-47df-8bfd-13c940c8a85b service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:34.403136 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e27a7c-f387-47df-8bfd-13c940c8a85b service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:34.403425 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e27a7c-f387-47df-8bfd-13c940c8a85b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:34.417217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 859/1719] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Aug 1 14:08:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:34.420416 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352d08a-d084-4692-8d8a-16e021acaefa None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:34.579719 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e27a7c-f387-47df-8bfd-13c940c8a85b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:34.583322 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e27a7c-f387-47df-8bfd-13c940c8a85b service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:34.598195 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352d08a-d084-4692-8d8a-16e021acaefa None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:34.610600 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f16fe-954c-494a-a5af-69d677c7e2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:34.611256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f16fe-954c-494a-a5af-69d677c7e2b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:34.611540 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f16fe-954c-494a-a5af-69d677c7e2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:34.611656 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 861/1720] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:34.611815 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f16fe-954c-494a-a5af-69d677c7e2b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:34.612668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f16fe-954c-494a-a5af-69d677c7e2b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:34.614097 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2582d9-c21d-4679-b1a2-7085609e4008 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:34.615475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2582d9-c21d-4679-b1a2-7085609e4008 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:34.615830 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2582d9-c21d-4679-b1a2-7085609e4008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:34.619473 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f16fe-954c-494a-a5af-69d677c7e2b8 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:34.666167 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 860/1721] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:34 2022] POST /identity/v3/projects => generated 385 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:34.670083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f16fe-954c-494a-a5af-69d677c7e2b8 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:34.710027 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-0c2582d9-c21d-4679-b1a2-7085609e4008 None None] MFA Rules not processed for user `3f61e3dd7eb1459381df05af41987bea`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:34.720402 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0c2582d9-c21d-4679-b1a2-7085609e4008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:34.834543 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f16fe-954c-494a-a5af-69d677c7e2b8 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:34.849026 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f1b12-6672-41f0-8d3b-5e769ded831b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:34.849851 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f1b12-6672-41f0-8d3b-5e769ded831b service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:34.850149 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f1b12-6672-41f0-8d3b-5e769ded831b service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:34.850425 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f1b12-6672-41f0-8d3b-5e769ded831b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:34.860913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:34.875253 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:34.879637 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8f1b12-6672-41f0-8d3b-5e769ded831b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:34.881350 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8f1b12-6672-41f0-8d3b-5e769ded831b service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:34.893892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 861/1722] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:34.894801 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f1b12-6672-41f0-8d3b-5e769ded831b service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:34.965856 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 862/1723] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:34 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:34.965856 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2582d9-c21d-4679-b1a2-7085609e4008 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:34.981215 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0c2582d9-c21d-4679-b1a2-7085609e4008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:35.079037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f1b12-6672-41f0-8d3b-5e769ded831b service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:35.103664 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e917deeb-1edc-4ff2-a3dc-8425b948be64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:35.104773 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917deeb-1edc-4ff2-a3dc-8425b948be64 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:35.105855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917deeb-1edc-4ff2-a3dc-8425b948be64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:35.105855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917deeb-1edc-4ff2-a3dc-8425b948be64 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:35.109476 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e917deeb-1edc-4ff2-a3dc-8425b948be64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:35.114392 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e917deeb-1edc-4ff2-a3dc-8425b948be64 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:35.157974 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 862/1724] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:34 2022] POST /identity/v3/users => generated 293 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:35.172552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e917deeb-1edc-4ff2-a3dc-8425b948be64 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:35.178787 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e917deeb-1edc-4ff2-a3dc-8425b948be64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:35.211037 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2582d9-c21d-4679-b1a2-7085609e4008 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:35.218772 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5a2d2-a162-486b-a3bc-9019d92e1ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:35.219840 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5a2d2-a162-486b-a3bc-9019d92e1ac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:35.220123 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5a2d2-a162-486b-a3bc-9019d92e1ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:35.221888 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5a2d2-a162-486b-a3bc-9019d92e1ac3 admin admin] PATH_INFO: `/v3/users/34fc070a8a7841db8bc221a427179d1c` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:35.224079 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db5a2d2-a162-486b-a3bc-9019d92e1ac3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34fc070a8a7841db8bc221a427179d1c)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:35.228679 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db5a2d2-a162-486b-a3bc-9019d92e1ac3 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:35.386110 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6db5a2d2-a162-486b-a3bc-9019d92e1ac3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34fc070a8a7841db8bc221a427179d1c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:35.388085 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6db5a2d2-a162-486b-a3bc-9019d92e1ac3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34fc070a8a7841db8bc221a427179d1c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:35.417930 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e917deeb-1edc-4ff2-a3dc-8425b948be64 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:35.430618 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6db5a2d2-a162-486b-a3bc-9019d92e1ac3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34fc070a8a7841db8bc221a427179d1c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:35.431672 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb902f2-8b3a-44c2-bc25-8305968fc8b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:35.432572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb902f2-8b3a-44c2-bc25-8305968fc8b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:35.433061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb902f2-8b3a-44c2-bc25-8305968fc8b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:35.433492 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb902f2-8b3a-44c2-bc25-8305968fc8b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:35.457204 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6db5a2d2-a162-486b-a3bc-9019d92e1ac3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34fc070a8a7841db8bc221a427179d1c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:35.458697 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb902f2-8b3a-44c2-bc25-8305968fc8b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:35.459037 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6db5a2d2-a162-486b-a3bc-9019d92e1ac3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34fc070a8a7841db8bc221a427179d1c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:35.460301 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb902f2-8b3a-44c2-bc25-8305968fc8b7 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:35.484333 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 863/1725] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:35 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:35.498905 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb902f2-8b3a-44c2-bc25-8305968fc8b7 service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:35.523679 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 863/1726] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:08:34 2022] DELETE /identity/v3/users/34fc070a8a7841db8bc221a427179d1c => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:35.531099 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e9270-bce9-4068-8672-5ee24a5f2585 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:35.532119 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e9270-bce9-4068-8672-5ee24a5f2585 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:35.538256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb902f2-8b3a-44c2-bc25-8305968fc8b7 service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:35.557362 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e9270-bce9-4068-8672-5ee24a5f2585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:35.650788 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-269e9270-bce9-4068-8672-5ee24a5f2585 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:35.658421 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-269e9270-bce9-4068-8672-5ee24a5f2585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:35.776565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab1f8b-a88b-43d3-8b09-d00232116c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:35.780069 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab1f8b-a88b-43d3-8b09-d00232116c14 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:35.781071 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab1f8b-a88b-43d3-8b09-d00232116c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:35.781343 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab1f8b-a88b-43d3-8b09-d00232116c14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:35.783342 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dab1f8b-a88b-43d3-8b09-d00232116c14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:35.790805 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dab1f8b-a88b-43d3-8b09-d00232116c14 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:35.838344 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 864/1727] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:35.839708 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab1f8b-a88b-43d3-8b09-d00232116c14 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:35.844307 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7dab1f8b-a88b-43d3-8b09-d00232116c14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:35.873846 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:35.883343 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab1f8b-a88b-43d3-8b09-d00232116c14 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:35.906262 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7a6e5-0d06-4246-b2a0-a88b6666351a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:35.910115 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7a6e5-0d06-4246-b2a0-a88b6666351a admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:35.910825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 864/1728] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:08:35.911083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7a6e5-0d06-4246-b2a0-a88b6666351a admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:35.911429 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7a6e5-0d06-4246-b2a0-a88b6666351a admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:35.912260 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba7a6e5-0d06-4246-b2a0-a88b6666351a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:35.914201 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba7a6e5-0d06-4246-b2a0-a88b6666351a admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:35.952628 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 865/1729] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:35.958052 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e9270-bce9-4068-8672-5ee24a5f2585 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:35.966864 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7a6e5-0d06-4246-b2a0-a88b6666351a admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:35.970885 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-cba7a6e5-0d06-4246-b2a0-a88b6666351a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:35.987285 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e9270-bce9-4068-8672-5ee24a5f2585 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:36.035273 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccce0f6-75ec-403e-9334-56ba3eea734b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:36.040449 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccce0f6-75ec-403e-9334-56ba3eea734b admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:36.041188 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccce0f6-75ec-403e-9334-56ba3eea734b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:36.041188 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccce0f6-75ec-403e-9334-56ba3eea734b admin admin] PATH_INFO: `/v3/projects/8ee6ba162508458fb76074fb13f161ca/users/e14cb0cb0b364747bc3e812a79a88911/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:36.048801 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:36.114564 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccce0f6-75ec-403e-9334-56ba3eea734b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ee6ba162508458fb76074fb13f161ca, user_id=e14cb0cb0b364747bc3e812a79a88911, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:36.118836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccce0f6-75ec-403e-9334-56ba3eea734b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:36.198133 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 865/1730] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:35 2022] PUT /identity/v3/projects/8ee6ba162508458fb76074fb13f161ca/users/e14cb0cb0b364747bc3e812a79a88911/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:36.204556 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91534e4b-050a-4baa-bdfb-198f07f62d7e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:36.205082 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91534e4b-050a-4baa-bdfb-198f07f62d7e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:36.205425 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7a6e5-0d06-4246-b2a0-a88b6666351a admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:36.205908 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91534e4b-050a-4baa-bdfb-198f07f62d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:36.269985 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-91534e4b-050a-4baa-bdfb-198f07f62d7e None None] MFA Rules not processed for user `e14cb0cb0b364747bc3e812a79a88911`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:36.273942 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-91534e4b-050a-4baa-bdfb-198f07f62d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:36.402662 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1f6203-9d72-493a-8aa6-087cee8f953c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:36.403835 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f6203-9d72-493a-8aa6-087cee8f953c service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:36.404370 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f6203-9d72-493a-8aa6-087cee8f953c service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:36.407895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f6203-9d72-493a-8aa6-087cee8f953c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:36.508013 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 866/1731] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:08:36.512918 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa642f-f663-4f99-b8a6-6cc6a2153b51 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:36.513271 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa642f-f663-4f99-b8a6-6cc6a2153b51 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:36.513499 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa642f-f663-4f99-b8a6-6cc6a2153b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:36.620435 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-81fa642f-f663-4f99-b8a6-6cc6a2153b51 None None] MFA Rules not processed for user `87f35e960544403ea30ba36d878a86cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:36.629876 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-81fa642f-f663-4f99-b8a6-6cc6a2153b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:36.739480 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1f6203-9d72-493a-8aa6-087cee8f953c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:36.742085 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1f6203-9d72-493a-8aa6-087cee8f953c service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:36.772360 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 866/1732] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 806 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:36.776362 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1f6203-9d72-493a-8aa6-087cee8f953c service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:36.797264 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1f6203-9d72-493a-8aa6-087cee8f953c service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:36.821373 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63271753-9758-4978-9bd6-de66f9556304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:36.822052 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63271753-9758-4978-9bd6-de66f9556304 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:36.822331 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63271753-9758-4978-9bd6-de66f9556304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:36.822634 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63271753-9758-4978-9bd6-de66f9556304 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:36.823260 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63271753-9758-4978-9bd6-de66f9556304 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:36.830068 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63271753-9758-4978-9bd6-de66f9556304 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:36.835962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 867/1733] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:36.839880 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa642f-f663-4f99-b8a6-6cc6a2153b51 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:36.845267 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-81fa642f-f663-4f99-b8a6-6cc6a2153b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:36.883028 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa642f-f663-4f99-b8a6-6cc6a2153b51 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:36.899790 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6be558-15c6-4eef-a4aa-6ab1ec2b08ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:36.901116 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6be558-15c6-4eef-a4aa-6ab1ec2b08ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:36.901410 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6be558-15c6-4eef-a4aa-6ab1ec2b08ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:36.901764 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6be558-15c6-4eef-a4aa-6ab1ec2b08ea admin admin] PATH_INFO: `/v3/projects/a72b3bf493df4e3a92824bd41c12c6e7` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:36.907191 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6be558-15c6-4eef-a4aa-6ab1ec2b08ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=a72b3bf493df4e3a92824bd41c12c6e7)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:36.909545 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6be558-15c6-4eef-a4aa-6ab1ec2b08ea admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:36.961850 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 867/1734] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:36 2022] POST /identity/v3/projects => generated 383 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:36.968159 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63271753-9758-4978-9bd6-de66f9556304 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:37.088289 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ed6be558-15c6-4eef-a4aa-6ab1ec2b08ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a72b3bf493df4e3a92824bd41c12c6e7', 'request_id': 'req-ed6be558-15c6-4eef-a4aa-6ab1ec2b08ea'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:37.111757 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ed6be558-15c6-4eef-a4aa-6ab1ec2b08ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a72b3bf493df4e3a92824bd41c12c6e7', 'request_id': 'req-ed6be558-15c6-4eef-a4aa-6ab1ec2b08ea'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:37.138501 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ed6be558-15c6-4eef-a4aa-6ab1ec2b08ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a72b3bf493df4e3a92824bd41c12c6e7', 'request_id': 'req-ed6be558-15c6-4eef-a4aa-6ab1ec2b08ea'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:37.144523 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:37.188612 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ed6be558-15c6-4eef-a4aa-6ab1ec2b08ea admin admin] The token cache is being invalidate because project a72b3bf493df4e3a92824bd41c12c6e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:08:37.189201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ed6be558-15c6-4eef-a4aa-6ab1ec2b08ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:37.193033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ed6be558-15c6-4eef-a4aa-6ab1ec2b08ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:37.200262 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 868/1735] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:08:36 2022] DELETE /identity/v3/projects/a72b3bf493df4e3a92824bd41c12c6e7 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:37.206263 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03134f68-7ff2-4022-8408-933b5b92080f None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:37.206754 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03134f68-7ff2-4022-8408-933b5b92080f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:37.207206 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03134f68-7ff2-4022-8408-933b5b92080f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:37.222347 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:37.239211 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63271753-9758-4978-9bd6-de66f9556304 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:37.313051 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-03134f68-7ff2-4022-8408-933b5b92080f None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:37.318225 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-03134f68-7ff2-4022-8408-933b5b92080f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:37.487928 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e482ec-70be-4a52-bcc1-be6189b6f7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:37.488934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e482ec-70be-4a52-bcc1-be6189b6f7c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:37.489416 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e482ec-70be-4a52-bcc1-be6189b6f7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:37.490153 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e482ec-70be-4a52-bcc1-be6189b6f7c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:37.491306 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e482ec-70be-4a52-bcc1-be6189b6f7c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:37.495960 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e482ec-70be-4a52-bcc1-be6189b6f7c3 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:37.535222 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 868/1736] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:36 2022] POST /identity/v3/users => generated 298 bytes in 568 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:37.537044 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be297111-f83b-455d-96ae-d70bf63846cd None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:37.537491 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be297111-f83b-455d-96ae-d70bf63846cd None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:37.537931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be297111-f83b-455d-96ae-d70bf63846cd None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:37.540428 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 869/1737] 10.222.0.45 () {54 vars in 961 bytes} [Mon Aug 1 14:08:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:08:37.544153 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be297111-f83b-455d-96ae-d70bf63846cd None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:37.546532 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-be297111-f83b-455d-96ae-d70bf63846cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:37.603178 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 869/1738] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:08:37.603178 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ca638-2d93-4e32-8a8f-a7feaf684917 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:37.603178 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ca638-2d93-4e32-8a8f-a7feaf684917 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:37.603178 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ca638-2d93-4e32-8a8f-a7feaf684917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:37.650551 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be297111-f83b-455d-96ae-d70bf63846cd None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:37.664474 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-f14ca638-2d93-4e32-8a8f-a7feaf684917 None None] MFA Rules not processed for user `58380e30789041e4ad9e8652bc628658`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:37.667011 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f14ca638-2d93-4e32-8a8f-a7feaf684917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:37.668479 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b42d7-e2d8-44a8-a294-5d1fc6ec276e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:37.669339 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b42d7-e2d8-44a8-a294-5d1fc6ec276e admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:37.669745 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b42d7-e2d8-44a8-a294-5d1fc6ec276e admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:37.670168 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b42d7-e2d8-44a8-a294-5d1fc6ec276e admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:37.671422 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b42d7-e2d8-44a8-a294-5d1fc6ec276e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:37.677453 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b42d7-e2d8-44a8-a294-5d1fc6ec276e admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:37.732823 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 870/1739] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:37.734271 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f388166-5c2d-4f23-b55f-4800218bd747 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:37.734569 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f388166-5c2d-4f23-b55f-4800218bd747 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:37.740639 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f388166-5c2d-4f23-b55f-4800218bd747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:37.806833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-7f388166-5c2d-4f23-b55f-4800218bd747 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:37.808200 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7f388166-5c2d-4f23-b55f-4800218bd747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:37.849792 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 871/1740] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:37.867365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f388166-5c2d-4f23-b55f-4800218bd747 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:37.910274 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f388166-5c2d-4f23-b55f-4800218bd747 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:37.958908 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b41dbf-1e2e-4568-8a85-6948263834a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:37.962366 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b41dbf-1e2e-4568-8a85-6948263834a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:37.963893 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b41dbf-1e2e-4568-8a85-6948263834a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:37.965156 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b41dbf-1e2e-4568-8a85-6948263834a1 admin admin] PATH_INFO: `/v3/projects/70d967fb1d25472fbb05e4269f78d4d0/users/5c9209d97cc644049618043b70ab1cba/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:38.013521 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 870/1741] 10.222.0.45 () {58 vars in 1058 bytes} [Mon Aug 1 14:08:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 416 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:38.018998 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ca638-2d93-4e32-8a8f-a7feaf684917 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:38.074203 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b41dbf-1e2e-4568-8a85-6948263834a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70d967fb1d25472fbb05e4269f78d4d0, user_id=5c9209d97cc644049618043b70ab1cba, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:38.074203 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b41dbf-1e2e-4568-8a85-6948263834a1 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:38.080463 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ca638-2d93-4e32-8a8f-a7feaf684917 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:38.104570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a536886-72ca-4030-9f6b-4a9a6de92032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:38.105719 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a536886-72ca-4030-9f6b-4a9a6de92032 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:38.106199 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a536886-72ca-4030-9f6b-4a9a6de92032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:38.107211 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a536886-72ca-4030-9f6b-4a9a6de92032 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:38.111169 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a536886-72ca-4030-9f6b-4a9a6de92032 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:38.111710 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a536886-72ca-4030-9f6b-4a9a6de92032 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:38.116315 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a536886-72ca-4030-9f6b-4a9a6de92032 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:38.156628 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 872/1742] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:37 2022] PUT /identity/v3/projects/70d967fb1d25472fbb05e4269f78d4d0/users/5c9209d97cc644049618043b70ab1cba/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:38.156628 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b41dbf-1e2e-4568-8a85-6948263834a1 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:38.202587 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 871/1743] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:38.212098 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a536886-72ca-4030-9f6b-4a9a6de92032 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:38.245027 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a536886-72ca-4030-9f6b-4a9a6de92032 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:38.423851 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b41dbf-1e2e-4568-8a85-6948263834a1 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:38.430435 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa81005e-159b-40c1-91a9-3aca0ce271be service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58380e30789041e4ad9e8652bc628658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:38.431981 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa81005e-159b-40c1-91a9-3aca0ce271be service placement] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:38.432552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa81005e-159b-40c1-91a9-3aca0ce271be service placement] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:38.433716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa81005e-159b-40c1-91a9-3aca0ce271be service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:38.437772 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-fa81005e-159b-40c1-91a9-3aca0ce271be service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:38.458075 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa81005e-159b-40c1-91a9-3aca0ce271be service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:38.461083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa81005e-159b-40c1-91a9-3aca0ce271be service placement] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:38.470549 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea7bc7-b408-4027-b477-b325107ebcfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:38.472656 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea7bc7-b408-4027-b477-b325107ebcfd admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:38.473298 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea7bc7-b408-4027-b477-b325107ebcfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:38.473937 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea7bc7-b408-4027-b477-b325107ebcfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:38.475350 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ea7bc7-b408-4027-b477-b325107ebcfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:38.478942 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ea7bc7-b408-4027-b477-b325107ebcfd admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:38.489606 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 873/1744] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:38.493949 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa81005e-159b-40c1-91a9-3aca0ce271be service placement] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:38.521780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa81005e-159b-40c1-91a9-3aca0ce271be service placement] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:38.526843 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 872/1745] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:38.530051 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3adfd8-f20c-483e-a323-2cf29ddf6733 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:38.537021 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3adfd8-f20c-483e-a323-2cf29ddf6733 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:38.537679 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3adfd8-f20c-483e-a323-2cf29ddf6733 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:38.544116 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 873/1746] 10.222.0.45 () {54 vars in 912 bytes} [Mon Aug 1 14:08:38 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:08:38.549307 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae5e26-0e4f-4ecb-95f2-95beb89a4c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:38.550023 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae5e26-0e4f-4ecb-95f2-95beb89a4c46 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:38.550304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae5e26-0e4f-4ecb-95f2-95beb89a4c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:38.550576 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae5e26-0e4f-4ecb-95f2-95beb89a4c46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:38.551272 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ae5e26-0e4f-4ecb-95f2-95beb89a4c46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:38.557717 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3adfd8-f20c-483e-a323-2cf29ddf6733 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:38.563252 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ae5e26-0e4f-4ecb-95f2-95beb89a4c46 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:38.631869 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3adfd8-f20c-483e-a323-2cf29ddf6733 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:38.637952 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 874/1747] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:38 2022] POST /identity/v3/projects => generated 365 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:38.637952 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa6a41-3744-4e75-9b4d-1232ae040473 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:38.637952 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa6a41-3744-4e75-9b4d-1232ae040473 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:38.637952 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa6a41-3744-4e75-9b4d-1232ae040473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:38.711065 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:38.717568 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-74fa6a41-3744-4e75-9b4d-1232ae040473 None None] MFA Rules not processed for user `6dd44b7a78594288b6aabb393fd6324a`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:38.719001 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-74fa6a41-3744-4e75-9b4d-1232ae040473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:38.787819 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:38.967819 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 875/1748] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Aug 1 14:08:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:38.969820 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa6a41-3744-4e75-9b4d-1232ae040473 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:39.041435 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b756b7fe-815f-44e8-aafc-08fbe1e11a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:39.042660 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756b7fe-815f-44e8-aafc-08fbe1e11a8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:39.043112 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756b7fe-815f-44e8-aafc-08fbe1e11a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:39.043569 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756b7fe-815f-44e8-aafc-08fbe1e11a8e admin admin] PATH_INFO: `/v3/projects/70d967fb1d25472fbb05e4269f78d4d0/users/5c9209d97cc644049618043b70ab1cba/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:39.061666 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b756b7fe-815f-44e8-aafc-08fbe1e11a8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=70d967fb1d25472fbb05e4269f78d4d0, user_id=5c9209d97cc644049618043b70ab1cba, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:39.066715 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b756b7fe-815f-44e8-aafc-08fbe1e11a8e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:39.125639 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 874/1749] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:38 2022] PUT /identity/v3/projects/70d967fb1d25472fbb05e4269f78d4d0/users/5c9209d97cc644049618043b70ab1cba/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:39.127749 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b756b7fe-815f-44e8-aafc-08fbe1e11a8e admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:39.140468 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b756b7fe-815f-44e8-aafc-08fbe1e11a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:39.317170 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa6a41-3744-4e75-9b4d-1232ae040473 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:39.366420 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b756b7fe-815f-44e8-aafc-08fbe1e11a8e admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:39.401773 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c78b4-ef13-4075-b21b-c42be7dd8c27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:39.402831 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c78b4-ef13-4075-b21b-c42be7dd8c27 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:39.403123 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c78b4-ef13-4075-b21b-c42be7dd8c27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:39.404683 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c78b4-ef13-4075-b21b-c42be7dd8c27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:39.418123 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c78b4-ef13-4075-b21b-c42be7dd8c27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:39.420585 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c78b4-ef13-4075-b21b-c42be7dd8c27 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:39.433455 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 875/1750] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:39.436758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7dfab1-038d-44f2-965f-a07be8365df5 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:39.439913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7dfab1-038d-44f2-965f-a07be8365df5 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:39.440964 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7dfab1-038d-44f2-965f-a07be8365df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:39.460232 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b477b6-a451-4a08-8000-069eaa813636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:39.461034 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b477b6-a451-4a08-8000-069eaa813636 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:39.461331 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b477b6-a451-4a08-8000-069eaa813636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:39.461615 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b477b6-a451-4a08-8000-069eaa813636 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:39.462338 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b477b6-a451-4a08-8000-069eaa813636 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:39.464527 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b477b6-a451-4a08-8000-069eaa813636 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:39.510509 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8c7dfab1-038d-44f2-965f-a07be8365df5 None None] MFA Rules not processed for user `5c9209d97cc644049618043b70ab1cba`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:39.514303 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 876/1751] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:38 2022] POST /identity/v3/users => generated 283 bytes in 543 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:39.514443 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8c7dfab1-038d-44f2-965f-a07be8365df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:39.514561 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b477b6-a451-4a08-8000-069eaa813636 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:39.517627 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-97b477b6-a451-4a08-8000-069eaa813636 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:39.556940 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b477b6-a451-4a08-8000-069eaa813636 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:39.561892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac5007c-cdc5-4098-a586-b2272633bb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:39.562902 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5007c-cdc5-4098-a586-b2272633bb56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:39.563409 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5007c-cdc5-4098-a586-b2272633bb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:39.563884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5007c-cdc5-4098-a586-b2272633bb56 admin admin] PATH_INFO: `/v3/projects/b42795ad0ec9407197883baa06958c59` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:39.565499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac5007c-cdc5-4098-a586-b2272633bb56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b42795ad0ec9407197883baa06958c59)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:39.567687 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac5007c-cdc5-4098-a586-b2272633bb56 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:39.670304 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 876/1752] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:39 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:39.671562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4af93-2f38-4b7e-bc40-f0305f93ad39 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:39.674063 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4af93-2f38-4b7e-bc40-f0305f93ad39 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:39.674388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4af93-2f38-4b7e-bc40-f0305f93ad39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:39.689278 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8ac5007c-cdc5-4098-a586-b2272633bb56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b42795ad0ec9407197883baa06958c59', 'request_id': 'req-8ac5007c-cdc5-4098-a586-b2272633bb56'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:39.700423 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8ac5007c-cdc5-4098-a586-b2272633bb56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b42795ad0ec9407197883baa06958c59', 'request_id': 'req-8ac5007c-cdc5-4098-a586-b2272633bb56'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:39.712386 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8ac5007c-cdc5-4098-a586-b2272633bb56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b42795ad0ec9407197883baa06958c59', 'request_id': 'req-8ac5007c-cdc5-4098-a586-b2272633bb56'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:39.714446 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:39.736314 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8ac5007c-cdc5-4098-a586-b2272633bb56 admin admin] The token cache is being invalidate because project b42795ad0ec9407197883baa06958c59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:08:39.737830 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8ac5007c-cdc5-4098-a586-b2272633bb56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:39.739969 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8ac5007c-cdc5-4098-a586-b2272633bb56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:39.740241 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-34a4af93-2f38-4b7e-bc40-f0305f93ad39 None None] MFA Rules not processed for user `a3e93dfa85244133b1641a431994b47f`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:39.741908 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-34a4af93-2f38-4b7e-bc40-f0305f93ad39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:39.747219 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 877/1753] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:08:39 2022] DELETE /identity/v3/projects/b42795ad0ec9407197883baa06958c59 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:39.749090 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac5007c-cdc5-4098-a586-b2272633bb56 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:39.751686 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-8ac5007c-cdc5-4098-a586-b2272633bb56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:39.806149 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:39.917126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 877/1754] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:39.917126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4af93-2f38-4b7e-bc40-f0305f93ad39 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:39.919656 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-34a4af93-2f38-4b7e-bc40-f0305f93ad39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:39.956225 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac5007c-cdc5-4098-a586-b2272633bb56 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:39.963108 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86006ef3-7906-4d25-8e3e-a3b04079799b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:39.963930 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86006ef3-7906-4d25-8e3e-a3b04079799b admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:39.964360 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86006ef3-7906-4d25-8e3e-a3b04079799b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:39.964737 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86006ef3-7906-4d25-8e3e-a3b04079799b admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:39.965971 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86006ef3-7906-4d25-8e3e-a3b04079799b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:39.968076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86006ef3-7906-4d25-8e3e-a3b04079799b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:39.987791 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 878/1755] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:39.989103 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86006ef3-7906-4d25-8e3e-a3b04079799b admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:39.992085 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-86006ef3-7906-4d25-8e3e-a3b04079799b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:40.055248 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86006ef3-7906-4d25-8e3e-a3b04079799b admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:40.071062 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fbd027-c0b3-4f79-98f6-d4a7b59fca8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:40.072393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbd027-c0b3-4f79-98f6-d4a7b59fca8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:40.075654 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbd027-c0b3-4f79-98f6-d4a7b59fca8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:40.076450 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbd027-c0b3-4f79-98f6-d4a7b59fca8d admin admin] PATH_INFO: `/v3/users/2e07e0829593453a81baa190e0c38b39` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:40.085327 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fbd027-c0b3-4f79-98f6-d4a7b59fca8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e07e0829593453a81baa190e0c38b39)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:40.087437 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fbd027-c0b3-4f79-98f6-d4a7b59fca8d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:40.101360 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4af93-2f38-4b7e-bc40-f0305f93ad39 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:40.106927 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71617470-3da2-480c-8fa7-13946cc0a533 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:40.107660 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71617470-3da2-480c-8fa7-13946cc0a533 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:40.108093 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71617470-3da2-480c-8fa7-13946cc0a533 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:40.108485 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71617470-3da2-480c-8fa7-13946cc0a533 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:40.111696 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-71617470-3da2-480c-8fa7-13946cc0a533 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:40.228787 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d9fbd027-c0b3-4f79-98f6-d4a7b59fca8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e07e0829593453a81baa190e0c38b39'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:40.230910 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d9fbd027-c0b3-4f79-98f6-d4a7b59fca8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e07e0829593453a81baa190e0c38b39'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:40.242244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d9fbd027-c0b3-4f79-98f6-d4a7b59fca8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e07e0829593453a81baa190e0c38b39'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:40.260574 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d9fbd027-c0b3-4f79-98f6-d4a7b59fca8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e07e0829593453a81baa190e0c38b39'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:40.270150 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71617470-3da2-480c-8fa7-13946cc0a533 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:40.273542 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71617470-3da2-480c-8fa7-13946cc0a533 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:40.285828 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 878/1756] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:40.287296 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c5c4e-7eb6-4f7a-910b-d725fa01aa25 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:40.287566 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c5c4e-7eb6-4f7a-910b-d725fa01aa25 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:40.287847 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c5c4e-7eb6-4f7a-910b-d725fa01aa25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:40.305179 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d9fbd027-c0b3-4f79-98f6-d4a7b59fca8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e07e0829593453a81baa190e0c38b39'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:40.319904 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-5f1c5c4e-7eb6-4f7a-910b-d725fa01aa25 None None] MFA Rules not processed for user `a3e93dfa85244133b1641a431994b47f`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:40.321509 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5f1c5c4e-7eb6-4f7a-910b-d725fa01aa25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:40.342701 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 879/1757] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:08:39 2022] DELETE /identity/v3/users/2e07e0829593453a81baa190e0c38b39 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:40.358735 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fbd027-c0b3-4f79-98f6-d4a7b59fca8d admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:40.359878 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 879/1758] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:40.365798 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d9fbd027-c0b3-4f79-98f6-d4a7b59fca8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:40.366188 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c5c4e-7eb6-4f7a-910b-d725fa01aa25 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:40.367655 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5f1c5c4e-7eb6-4f7a-910b-d725fa01aa25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:40.571894 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c5c4e-7eb6-4f7a-910b-d725fa01aa25 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:40.594207 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:40.628041 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:40.637196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fbd027-c0b3-4f79-98f6-d4a7b59fca8d admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:40.651145 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b3d76-fe71-489a-b561-e776b69f759d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:40.661668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b3d76-fe71-489a-b561-e776b69f759d admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:40.662580 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b3d76-fe71-489a-b561-e776b69f759d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:40.662580 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b3d76-fe71-489a-b561-e776b69f759d admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:40.663077 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b3d76-fe71-489a-b561-e776b69f759d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:40.676042 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b3d76-fe71-489a-b561-e776b69f759d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:40.695576 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 880/1759] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:40.697184 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b3d76-fe71-489a-b561-e776b69f759d admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:40.708617 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b3d76-fe71-489a-b561-e776b69f759d admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:40.715143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7446c6-41e2-4736-b329-f3b25a67e038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:40.716079 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7446c6-41e2-4736-b329-f3b25a67e038 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:40.716831 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7446c6-41e2-4736-b329-f3b25a67e038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:40.717338 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7446c6-41e2-4736-b329-f3b25a67e038 admin admin] PATH_INFO: `/v3/projects/392d4b2368d24b35aa90c19524360810/users/76ca2cc4ff394733bc9c04d780527a7a/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:40.761580 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7446c6-41e2-4736-b329-f3b25a67e038 admin admin] RBAC: Authorizing `identity:create_grant(project_id=392d4b2368d24b35aa90c19524360810, user_id=76ca2cc4ff394733bc9c04d780527a7a, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:40.771408 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7446c6-41e2-4736-b329-f3b25a67e038 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:40.818431 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 881/1760] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:40 2022] PUT /identity/v3/projects/392d4b2368d24b35aa90c19524360810/users/76ca2cc4ff394733bc9c04d780527a7a/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:40.822583 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa9b53-844c-4267-9ee6-786542bef1e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:40.822799 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a455dce9-0d5b-4a80-b189-887c90047d01 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:40.823059 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a455dce9-0d5b-4a80-b189-887c90047d01 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:40.823302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a455dce9-0d5b-4a80-b189-887c90047d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:40.826761 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa9b53-844c-4267-9ee6-786542bef1e4 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:40.828728 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa9b53-844c-4267-9ee6-786542bef1e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:40.829231 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa9b53-844c-4267-9ee6-786542bef1e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:40.836253 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-06fa9b53-844c-4267-9ee6-786542bef1e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:40.911975 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-a455dce9-0d5b-4a80-b189-887c90047d01 None None] MFA Rules not processed for user `76ca2cc4ff394733bc9c04d780527a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:40.913525 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a455dce9-0d5b-4a80-b189-887c90047d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:41.105486 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fa9b53-844c-4267-9ee6-786542bef1e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:41.107140 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fa9b53-844c-4267-9ee6-786542bef1e4 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:41.137899 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 882/1761] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:40 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:41.138942 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a455dce9-0d5b-4a80-b189-887c90047d01 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:41.150437 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 880/1762] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 784 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:08:41.351767 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a455dce9-0d5b-4a80-b189-887c90047d01 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:41.372479 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ee6c3-bb31-461c-97f5-eb9fbfaed22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:41.383883 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ee6c3-bb31-461c-97f5-eb9fbfaed22f admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:41.385880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ee6c3-bb31-461c-97f5-eb9fbfaed22f admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:41.387104 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ee6c3-bb31-461c-97f5-eb9fbfaed22f admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:41.388792 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10ee6c3-bb31-461c-97f5-eb9fbfaed22f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:41.409910 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa9b53-844c-4267-9ee6-786542bef1e4 service nova] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:41.409910 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-06fa9b53-844c-4267-9ee6-786542bef1e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:41.410085 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10ee6c3-bb31-461c-97f5-eb9fbfaed22f admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:41.437018 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa9b53-844c-4267-9ee6-786542bef1e4 service nova] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:41.457811 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af4251-01f6-4fce-ae52-272275f1a0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:41.458540 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af4251-01f6-4fce-ae52-272275f1a0c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:41.458916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af4251-01f6-4fce-ae52-272275f1a0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:41.459381 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af4251-01f6-4fce-ae52-272275f1a0c2 admin admin] PATH_INFO: `/v3/projects/1aeb8523bfb541c8aca84fbe2954da03` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:41.462039 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af4251-01f6-4fce-ae52-272275f1a0c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1aeb8523bfb541c8aca84fbe2954da03)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:41.469729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af4251-01f6-4fce-ae52-272275f1a0c2 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:41.548480 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 883/1763] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:41 2022] POST /identity/v3/projects => generated 365 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:41.549565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ee6c3-bb31-461c-97f5-eb9fbfaed22f admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:41.640981 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-92af4251-01f6-4fce-ae52-272275f1a0c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1aeb8523bfb541c8aca84fbe2954da03', 'request_id': 'req-92af4251-01f6-4fce-ae52-272275f1a0c2'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:41.660714 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-92af4251-01f6-4fce-ae52-272275f1a0c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1aeb8523bfb541c8aca84fbe2954da03', 'request_id': 'req-92af4251-01f6-4fce-ae52-272275f1a0c2'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:41.678005 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-92af4251-01f6-4fce-ae52-272275f1a0c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1aeb8523bfb541c8aca84fbe2954da03', 'request_id': 'req-92af4251-01f6-4fce-ae52-272275f1a0c2'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:41.700981 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-92af4251-01f6-4fce-ae52-272275f1a0c2 admin admin] The token cache is being invalidate because project 1aeb8523bfb541c8aca84fbe2954da03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:08:41.701626 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-92af4251-01f6-4fce-ae52-272275f1a0c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:41.702576 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-92af4251-01f6-4fce-ae52-272275f1a0c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:41.705191 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 881/1764] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:08:41 2022] DELETE /identity/v3/projects/1aeb8523bfb541c8aca84fbe2954da03 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:41.711846 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421904fb-3be5-431e-9316-f6402917ec78 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:41.713549 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421904fb-3be5-431e-9316-f6402917ec78 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:41.714037 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421904fb-3be5-431e-9316-f6402917ec78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:41.744502 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:41.796383 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-421904fb-3be5-431e-9316-f6402917ec78 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:41.801710 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-421904fb-3be5-431e-9316-f6402917ec78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:41.826528 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ee6c3-bb31-461c-97f5-eb9fbfaed22f admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:41.866695 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:41.943829 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 882/1765] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:41.951179 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae1a5c-f8fa-4537-98ec-8ee5197ee44f None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:41.952982 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae1a5c-f8fa-4537-98ec-8ee5197ee44f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:41.953423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae1a5c-f8fa-4537-98ec-8ee5197ee44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:41.979997 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37271814-dd60-410d-9ca4-2a3b7e8ed677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:41.980694 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37271814-dd60-410d-9ca4-2a3b7e8ed677 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:41.981725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37271814-dd60-410d-9ca4-2a3b7e8ed677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:41.982867 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37271814-dd60-410d-9ca4-2a3b7e8ed677 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:41.986518 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37271814-dd60-410d-9ca4-2a3b7e8ed677 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:41.990332 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-4aae1a5c-f8fa-4537-98ec-8ee5197ee44f None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:41.990780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37271814-dd60-410d-9ca4-2a3b7e8ed677 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:41.992114 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4aae1a5c-f8fa-4537-98ec-8ee5197ee44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:42.002295 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 883/1766] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:42.010979 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae1a5c-f8fa-4537-98ec-8ee5197ee44f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:42.048423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae1a5c-f8fa-4537-98ec-8ee5197ee44f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:42.066034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e0e24-9ea9-4530-857a-519ce138cb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:42.066691 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e0e24-9ea9-4530-857a-519ce138cb81 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:42.067008 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e0e24-9ea9-4530-857a-519ce138cb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:42.067243 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e0e24-9ea9-4530-857a-519ce138cb81 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:42.067955 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e0e24-9ea9-4530-857a-519ce138cb81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:42.068264 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e0e24-9ea9-4530-857a-519ce138cb81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:42.069921 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e0e24-9ea9-4530-857a-519ce138cb81 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:42.071251 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 884/1767] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:41 2022] POST /identity/v3/users => generated 289 bytes in 522 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:42.076054 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37271814-dd60-410d-9ca4-2a3b7e8ed677 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:42.104651 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-37271814-dd60-410d-9ca4-2a3b7e8ed677 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:42.112459 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 884/1768] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:08:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:42.122625 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e0e24-9ea9-4530-857a-519ce138cb81 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:42.158858 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e0e24-9ea9-4530-857a-519ce138cb81 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:42.160566 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37271814-dd60-410d-9ca4-2a3b7e8ed677 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:42.172056 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d774a345-d30f-4d74-8818-00042489824d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:42.172917 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774a345-d30f-4d74-8818-00042489824d admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:42.173237 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774a345-d30f-4d74-8818-00042489824d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:42.173513 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774a345-d30f-4d74-8818-00042489824d admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:42.174794 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d774a345-d30f-4d74-8818-00042489824d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:42.177201 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d774a345-d30f-4d74-8818-00042489824d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:42.180152 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b8997b-d320-4ef6-8ae5-2901df8ef1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:42.180439 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8997b-d320-4ef6-8ae5-2901df8ef1b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:42.180763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8997b-d320-4ef6-8ae5-2901df8ef1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:42.181090 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8997b-d320-4ef6-8ae5-2901df8ef1b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:42.181748 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b8997b-d320-4ef6-8ae5-2901df8ef1b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:42.189108 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b8997b-d320-4ef6-8ae5-2901df8ef1b2 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:42.194382 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 885/1769] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:42.201525 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d774a345-d30f-4d74-8818-00042489824d admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:42.232952 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d774a345-d30f-4d74-8818-00042489824d admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:42.252421 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61996079-6551-46cc-a17d-6d603011acc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:42.253198 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61996079-6551-46cc-a17d-6d603011acc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:42.253477 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61996079-6551-46cc-a17d-6d603011acc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:42.253756 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61996079-6551-46cc-a17d-6d603011acc1 admin admin] PATH_INFO: `/v3/projects/1d5300c73af44847a15c9f459975b59d/users/9287ab95d897448cb3c7639db31310c0/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:42.269968 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 885/1770] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:42 2022] POST /identity/v3/projects => generated 385 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:42.272354 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b8997b-d320-4ef6-8ae5-2901df8ef1b2 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:42.281892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61996079-6551-46cc-a17d-6d603011acc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d5300c73af44847a15c9f459975b59d, user_id=9287ab95d897448cb3c7639db31310c0, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:42.285122 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61996079-6551-46cc-a17d-6d603011acc1 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:42.339034 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 886/1771] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:42 2022] PUT /identity/v3/projects/1d5300c73af44847a15c9f459975b59d/users/9287ab95d897448cb3c7639db31310c0/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:42.344630 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61996079-6551-46cc-a17d-6d603011acc1 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:42.488652 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61996079-6551-46cc-a17d-6d603011acc1 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:42.504576 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04780f-f5bb-447e-8185-19f46b20380f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:42.505700 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04780f-f5bb-447e-8185-19f46b20380f admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:42.507550 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04780f-f5bb-447e-8185-19f46b20380f admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:42.509400 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04780f-f5bb-447e-8185-19f46b20380f admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:42.510383 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc04780f-f5bb-447e-8185-19f46b20380f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:42.515533 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc04780f-f5bb-447e-8185-19f46b20380f admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:42.541536 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b8997b-d320-4ef6-8ae5-2901df8ef1b2 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:42.559411 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 887/1772] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:42.559411 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba613681-8a1e-42bc-8e00-2ad2d27546f4 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:42.560651 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f12fe12-84ed-4b50-bcb5-d80ac92f148b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:42.561541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12fe12-84ed-4b50-bcb5-d80ac92f148b admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:42.562024 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12fe12-84ed-4b50-bcb5-d80ac92f148b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:42.562143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba613681-8a1e-42bc-8e00-2ad2d27546f4 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:42.562591 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12fe12-84ed-4b50-bcb5-d80ac92f148b admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:42.562702 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba613681-8a1e-42bc-8e00-2ad2d27546f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:42.563553 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f12fe12-84ed-4b50-bcb5-d80ac92f148b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:42.571071 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f12fe12-84ed-4b50-bcb5-d80ac92f148b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:42.618645 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 886/1773] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:42 2022] POST /identity/v3/users => generated 293 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:42.619220 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f12fe12-84ed-4b50-bcb5-d80ac92f148b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:42.658974 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f12fe12-84ed-4b50-bcb5-d80ac92f148b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:42.674726 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb4c565-f177-443a-8a07-d0ca86331f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:42.675810 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4c565-f177-443a-8a07-d0ca86331f52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:42.676363 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4c565-f177-443a-8a07-d0ca86331f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:42.676833 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4c565-f177-443a-8a07-d0ca86331f52 admin admin] PATH_INFO: `/v3/projects/1d5300c73af44847a15c9f459975b59d/users/9287ab95d897448cb3c7639db31310c0/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:42.685349 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb4c565-f177-443a-8a07-d0ca86331f52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d5300c73af44847a15c9f459975b59d, user_id=9287ab95d897448cb3c7639db31310c0, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:42.689330 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb4c565-f177-443a-8a07-d0ca86331f52 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:42.718755 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 887/1774] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:42 2022] PUT /identity/v3/projects/1d5300c73af44847a15c9f459975b59d/users/9287ab95d897448cb3c7639db31310c0/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:42.720657 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb4c565-f177-443a-8a07-d0ca86331f52 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:42.752539 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-ba613681-8a1e-42bc-8e00-2ad2d27546f4 None None] MFA Rules not processed for user `e07e83c0bdba47d39c09308eba1bdaae`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:42.759448 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ba613681-8a1e-42bc-8e00-2ad2d27546f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:42.792549 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:42.872856 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb4c565-f177-443a-8a07-d0ca86331f52 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:42.884457 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a15c4-340b-4f31-9a98-6d424bdd6b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:42.885636 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a15c4-340b-4f31-9a98-6d424bdd6b3b admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:42.885923 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a15c4-340b-4f31-9a98-6d424bdd6b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:42.886201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a15c4-340b-4f31-9a98-6d424bdd6b3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:42.893827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20a15c4-340b-4f31-9a98-6d424bdd6b3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:42.898854 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20a15c4-340b-4f31-9a98-6d424bdd6b3b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:42.926097 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 888/1775] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:42.928792 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99da450-905b-412c-a9b9-b927e27dd79f None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:42.930465 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99da450-905b-412c-a9b9-b927e27dd79f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:42.930768 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99da450-905b-412c-a9b9-b927e27dd79f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:42.964073 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:42.998894 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 888/1776] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Aug 1 14:08:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 441 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:42.999855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba613681-8a1e-42bc-8e00-2ad2d27546f4 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:43.022445 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba613681-8a1e-42bc-8e00-2ad2d27546f4 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:43.029728 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ca5b9-7674-48a6-b6e8-0021e8e3be64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:43.039685 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ca5b9-7674-48a6-b6e8-0021e8e3be64 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:43.044596 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ca5b9-7674-48a6-b6e8-0021e8e3be64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:43.045244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ca5b9-7674-48a6-b6e8-0021e8e3be64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:43.046820 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ca5b9-7674-48a6-b6e8-0021e8e3be64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:43.049240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-e99da450-905b-412c-a9b9-b927e27dd79f None None] MFA Rules not processed for user `9287ab95d897448cb3c7639db31310c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:43.050578 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e99da450-905b-412c-a9b9-b927e27dd79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:43.052530 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ca5b9-7674-48a6-b6e8-0021e8e3be64 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:43.089378 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 889/1777] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:43.093375 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc915de-b4fc-4891-8773-6cf3833c0d0d None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:43.095614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc915de-b4fc-4891-8773-6cf3833c0d0d None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:43.095982 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc915de-b4fc-4891-8773-6cf3833c0d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:43.217242 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-efc915de-b4fc-4891-8773-6cf3833c0d0d None None] MFA Rules not processed for user `a3e93dfa85244133b1641a431994b47f`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:43.222308 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-efc915de-b4fc-4891-8773-6cf3833c0d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:43.306882 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 889/1778] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:42 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:43.308869 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99da450-905b-412c-a9b9-b927e27dd79f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:43.369136 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99da450-905b-412c-a9b9-b927e27dd79f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:43.412464 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f947e-99f7-45b0-8b05-f421eb60e1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:43.420301 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f947e-99f7-45b0-8b05-f421eb60e1c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:43.421152 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f947e-99f7-45b0-8b05-f421eb60e1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:43.421469 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f947e-99f7-45b0-8b05-f421eb60e1c4 admin admin] PATH_INFO: `/v3/projects/2b493738161d4f9289fc1ea2808c777e/users/2fc8c39f4d574eaab2459fc924b2486b/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:43.485742 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9f947e-99f7-45b0-8b05-f421eb60e1c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b493738161d4f9289fc1ea2808c777e, user_id=2fc8c39f4d574eaab2459fc924b2486b, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:43.494545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 890/1779] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:43.495453 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc915de-b4fc-4891-8773-6cf3833c0d0d None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:43.499335 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9f947e-99f7-45b0-8b05-f421eb60e1c4 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:43.507743 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-efc915de-b4fc-4891-8773-6cf3833c0d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:43.548652 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 890/1780] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:43 2022] PUT /identity/v3/projects/2b493738161d4f9289fc1ea2808c777e/users/2fc8c39f4d574eaab2459fc924b2486b/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:43.560723 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89e91b-a5a0-4021-a189-d14227cf894a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:43.561227 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89e91b-a5a0-4021-a189-d14227cf894a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:43.561644 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89e91b-a5a0-4021-a189-d14227cf894a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:43.648119 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-be89e91b-a5a0-4021-a189-d14227cf894a None None] MFA Rules not processed for user `2fc8c39f4d574eaab2459fc924b2486b`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:43.652953 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-be89e91b-a5a0-4021-a189-d14227cf894a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:43.900578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc915de-b4fc-4891-8773-6cf3833c0d0d None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:43.965947 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 891/1781] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:43.967011 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:43.969072 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be89e91b-a5a0-4021-a189-d14227cf894a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:43.971334 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-be89e91b-a5a0-4021-a189-d14227cf894a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:44.065929 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d96cc-aaf3-4c07-8ed8-413be5e36275 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cda7f365ba0405daaf86870960f12d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:44.067890 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d96cc-aaf3-4c07-8ed8-413be5e36275 service swift] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:44.068505 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d96cc-aaf3-4c07-8ed8-413be5e36275 service swift] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:44.069920 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d96cc-aaf3-4c07-8ed8-413be5e36275 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:44.083656 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be89e91b-a5a0-4021-a189-d14227cf894a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:44.096662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff01694-e0c2-448f-ab13-ebe1478a1656 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:44.097670 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff01694-e0c2-448f-ab13-ebe1478a1656 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:44.097998 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff01694-e0c2-448f-ab13-ebe1478a1656 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:44.098331 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff01694-e0c2-448f-ab13-ebe1478a1656 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:44.106007 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:44.217705 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff01694-e0c2-448f-ab13-ebe1478a1656 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:44.219774 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff01694-e0c2-448f-ab13-ebe1478a1656 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:44.247359 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d96cc-aaf3-4c07-8ed8-413be5e36275 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:44.253225 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 892/1782] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:43 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:44.257224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d96cc-aaf3-4c07-8ed8-413be5e36275 service swift] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:44.257473 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff01694-e0c2-448f-ab13-ebe1478a1656 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:44.309659 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 891/1783] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 14:08:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 794 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:08:44.517937 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b48f31-944f-47b4-ae2f-1f112cc3b244 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:44.519424 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b48f31-944f-47b4-ae2f-1f112cc3b244 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:44.519889 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b48f31-944f-47b4-ae2f-1f112cc3b244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:44.601311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-21b48f31-944f-47b4-ae2f-1f112cc3b244 None None] MFA Rules not processed for user `6dd44b7a78594288b6aabb393fd6324a`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:44.604650 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-21b48f31-944f-47b4-ae2f-1f112cc3b244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:44.607580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff01694-e0c2-448f-ab13-ebe1478a1656 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:44.634160 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed7fe6-5122-4482-9877-521067b84038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:44.634988 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed7fe6-5122-4482-9877-521067b84038 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:44.636245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed7fe6-5122-4482-9877-521067b84038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:44.636667 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed7fe6-5122-4482-9877-521067b84038 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:44.638529 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed7fe6-5122-4482-9877-521067b84038 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:44.644357 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed7fe6-5122-4482-9877-521067b84038 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:44.748876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 892/1784] 10.222.0.57 () {58 vars in 1015 bytes} [Mon Aug 1 14:08:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:44.782179 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 893/1785] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:44 2022] POST /identity/v3/projects => generated 385 bytes in 528 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:44.787163 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b48f31-944f-47b4-ae2f-1f112cc3b244 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:44.812484 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b48f31-944f-47b4-ae2f-1f112cc3b244 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:44.948884 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed7fe6-5122-4482-9877-521067b84038 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:44.953969 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-66ed7fe6-5122-4482-9877-521067b84038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:45.145765 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0051225-56d2-433a-bf10-8dcc3cb4629f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:45.146571 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0051225-56d2-433a-bf10-8dcc3cb4629f admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:45.147361 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0051225-56d2-433a-bf10-8dcc3cb4629f admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:45.153025 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0051225-56d2-433a-bf10-8dcc3cb4629f admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:45.153276 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0051225-56d2-433a-bf10-8dcc3cb4629f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:45.164607 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0051225-56d2-433a-bf10-8dcc3cb4629f admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:45.180575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed7fe6-5122-4482-9877-521067b84038 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:45.211456 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 893/1786] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:44 2022] POST /identity/v3/users => generated 299 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:45.216361 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0051225-56d2-433a-bf10-8dcc3cb4629f admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:45.218549 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf6c2f-055f-4cac-b8d7-36584fdb7d38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:45.219248 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf6c2f-055f-4cac-b8d7-36584fdb7d38 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:45.219554 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf6c2f-055f-4cac-b8d7-36584fdb7d38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:45.219813 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf6c2f-055f-4cac-b8d7-36584fdb7d38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:45.247484 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0051225-56d2-433a-bf10-8dcc3cb4629f admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:45.265743 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22838894-6085-419e-8096-cda5d0230ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:45.266971 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22838894-6085-419e-8096-cda5d0230ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:45.268555 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22838894-6085-419e-8096-cda5d0230ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:45.268871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22838894-6085-419e-8096-cda5d0230ad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:45.272133 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22838894-6085-419e-8096-cda5d0230ad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:45.279125 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22838894-6085-419e-8096-cda5d0230ad5 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:45.284511 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf6c2f-055f-4cac-b8d7-36584fdb7d38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:45.286726 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf6c2f-055f-4cac-b8d7-36584fdb7d38 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:45.308790 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:45.327212 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 894/1787] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:45.332976 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 894/1788] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:45.335708 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf6c2f-055f-4cac-b8d7-36584fdb7d38 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:45.374924 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf6c2f-055f-4cac-b8d7-36584fdb7d38 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:45.401992 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8501ec-c2aa-4228-a331-d1a27a642d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:45.403506 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8501ec-c2aa-4228-a331-d1a27a642d07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:45.405698 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8501ec-c2aa-4228-a331-d1a27a642d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:45.406089 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8501ec-c2aa-4228-a331-d1a27a642d07 admin admin] PATH_INFO: `/v3/projects/ec2cb6858b8f450aa72fc275d9c882fe/users/3e04c66ab4e44ded9494e9870e6c6732/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:45.469102 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8501ec-c2aa-4228-a331-d1a27a642d07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec2cb6858b8f450aa72fc275d9c882fe, user_id=3e04c66ab4e44ded9494e9870e6c6732, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:45.480072 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8501ec-c2aa-4228-a331-d1a27a642d07 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:45.526379 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 895/1789] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:45 2022] PUT /identity/v3/projects/ec2cb6858b8f450aa72fc275d9c882fe/users/3e04c66ab4e44ded9494e9870e6c6732/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:45.531058 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22838894-6085-419e-8096-cda5d0230ad5 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:45.549061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22838894-6085-419e-8096-cda5d0230ad5 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:45.764471 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7955553-ca6a-4166-beee-a682d644a7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:45.766220 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7955553-ca6a-4166-beee-a682d644a7b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:45.766615 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7955553-ca6a-4166-beee-a682d644a7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:45.767015 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7955553-ca6a-4166-beee-a682d644a7b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:45.768220 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7955553-ca6a-4166-beee-a682d644a7b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:45.772915 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7955553-ca6a-4166-beee-a682d644a7b4 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:45.828420 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 895/1790] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:45.833669 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8501ec-c2aa-4228-a331-d1a27a642d07 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:45.865339 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8501ec-c2aa-4228-a331-d1a27a642d07 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:45.873743 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f3db3a-2623-4e40-a857-88fa59d73028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:45.878200 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f3db3a-2623-4e40-a857-88fa59d73028 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:45.878946 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f3db3a-2623-4e40-a857-88fa59d73028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:45.879411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f3db3a-2623-4e40-a857-88fa59d73028 admin admin] PATH_INFO: `/v3/projects/ec2cb6858b8f450aa72fc275d9c882fe/users/3e04c66ab4e44ded9494e9870e6c6732/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:45.882813 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f3db3a-2623-4e40-a857-88fa59d73028 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec2cb6858b8f450aa72fc275d9c882fe, user_id=3e04c66ab4e44ded9494e9870e6c6732, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:45.887644 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f3db3a-2623-4e40-a857-88fa59d73028 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:45.893814 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:45.921040 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 896/1791] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:45 2022] PUT /identity/v3/projects/ec2cb6858b8f450aa72fc275d9c882fe/users/3e04c66ab4e44ded9494e9870e6c6732/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:45.923105 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475f776-4934-4008-b883-3ca29fe88df5 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:45.923427 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475f776-4934-4008-b883-3ca29fe88df5 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:45.923704 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475f776-4934-4008-b883-3ca29fe88df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:45.993127 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-9475f776-4934-4008-b883-3ca29fe88df5 None None] MFA Rules not processed for user `3e04c66ab4e44ded9494e9870e6c6732`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:45.994659 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-9475f776-4934-4008-b883-3ca29fe88df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:46.234909 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 896/1792] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:45 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:46.299201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f3db3a-2623-4e40-a857-88fa59d73028 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:46.303323 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475f776-4934-4008-b883-3ca29fe88df5 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:46.305510 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-9475f776-4934-4008-b883-3ca29fe88df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:46.737097 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475f776-4934-4008-b883-3ca29fe88df5 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:46.741065 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f3db3a-2623-4e40-a857-88fa59d73028 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:46.749813 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e096c13-8900-4e2e-80d7-d63b9b301fb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:46.750525 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e096c13-8900-4e2e-80d7-d63b9b301fb2 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:46.750812 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e096c13-8900-4e2e-80d7-d63b9b301fb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:46.751109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e096c13-8900-4e2e-80d7-d63b9b301fb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:46.768140 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44189dc-526c-4943-af9a-06d0100c14dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:46.769692 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44189dc-526c-4943-af9a-06d0100c14dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:46.769982 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44189dc-526c-4943-af9a-06d0100c14dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:46.770266 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44189dc-526c-4943-af9a-06d0100c14dd admin admin] PATH_INFO: `/v3/users/76ca2cc4ff394733bc9c04d780527a7a` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:46.772463 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44189dc-526c-4943-af9a-06d0100c14dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=76ca2cc4ff394733bc9c04d780527a7a)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:46.774518 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44189dc-526c-4943-af9a-06d0100c14dd admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:46.790491 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e096c13-8900-4e2e-80d7-d63b9b301fb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:46.800652 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e096c13-8900-4e2e-80d7-d63b9b301fb2 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:46.845414 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 897/1793] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:46 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:08:46.949630 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e096c13-8900-4e2e-80d7-d63b9b301fb2 service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:46.971170 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e096c13-8900-4e2e-80d7-d63b9b301fb2 service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:46.975544 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a44189dc-526c-4943-af9a-06d0100c14dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76ca2cc4ff394733bc9c04d780527a7a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:46.978835 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a44189dc-526c-4943-af9a-06d0100c14dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76ca2cc4ff394733bc9c04d780527a7a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:47.001221 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92d48a-4216-4dfd-ac4b-c6b4b1b70521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:47.006414 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92d48a-4216-4dfd-ac4b-c6b4b1b70521 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:47.009448 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92d48a-4216-4dfd-ac4b-c6b4b1b70521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:47.010843 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92d48a-4216-4dfd-ac4b-c6b4b1b70521 admin admin] PATH_INFO: `/v3/users/2fc8c39f4d574eaab2459fc924b2486b` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:47.016012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b92d48a-4216-4dfd-ac4b-c6b4b1b70521 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fc8c39f4d574eaab2459fc924b2486b)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:47.020825 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b92d48a-4216-4dfd-ac4b-c6b4b1b70521 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:47.021188 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a44189dc-526c-4943-af9a-06d0100c14dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76ca2cc4ff394733bc9c04d780527a7a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:47.034146 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a44189dc-526c-4943-af9a-06d0100c14dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76ca2cc4ff394733bc9c04d780527a7a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:47.035959 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a44189dc-526c-4943-af9a-06d0100c14dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76ca2cc4ff394733bc9c04d780527a7a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:47.060559 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 897/1794] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:08:46 2022] DELETE /identity/v3/users/76ca2cc4ff394733bc9c04d780527a7a => generated 0 bytes in 762 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:47.071395 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bff7b-f3f8-4682-a8fe-ffcccfd0fd92 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:47.072684 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bff7b-f3f8-4682-a8fe-ffcccfd0fd92 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:47.073312 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bff7b-f3f8-4682-a8fe-ffcccfd0fd92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:47.088325 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:47.127024 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-259bff7b-f3f8-4682-a8fe-ffcccfd0fd92 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:47.127024 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-259bff7b-f3f8-4682-a8fe-ffcccfd0fd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:47.160861 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1b92d48a-4216-4dfd-ac4b-c6b4b1b70521 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fc8c39f4d574eaab2459fc924b2486b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:47.164393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1b92d48a-4216-4dfd-ac4b-c6b4b1b70521 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fc8c39f4d574eaab2459fc924b2486b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:47.194578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1b92d48a-4216-4dfd-ac4b-c6b4b1b70521 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fc8c39f4d574eaab2459fc924b2486b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:47.213837 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1b92d48a-4216-4dfd-ac4b-c6b4b1b70521 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fc8c39f4d574eaab2459fc924b2486b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:47.239881 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1b92d48a-4216-4dfd-ac4b-c6b4b1b70521 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fc8c39f4d574eaab2459fc924b2486b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:47.248960 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 898/1795] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:08:46 2022] DELETE /identity/v3/users/2fc8c39f4d574eaab2459fc924b2486b => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:47.253794 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa4bb2-f9d1-4e2e-a28d-7e29c02e8236 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:47.254400 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa4bb2-f9d1-4e2e-a28d-7e29c02e8236 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:47.254881 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa4bb2-f9d1-4e2e-a28d-7e29c02e8236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:47.285993 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-ccaa4bb2-f9d1-4e2e-a28d-7e29c02e8236 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:47.288360 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ccaa4bb2-f9d1-4e2e-a28d-7e29c02e8236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:47.393641 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 898/1796] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:47.398123 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259bff7b-f3f8-4682-a8fe-ffcccfd0fd92 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:47.403290 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-259bff7b-f3f8-4682-a8fe-ffcccfd0fd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:47.470369 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:47.488740 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 899/1797] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:47.592664 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259bff7b-f3f8-4682-a8fe-ffcccfd0fd92 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:47.645369 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaa4bb2-f9d1-4e2e-a28d-7e29c02e8236 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:47.647048 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ccaa4bb2-f9d1-4e2e-a28d-7e29c02e8236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:47.730967 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6d416-b42c-4f77-b801-14682a3b7226 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:47.732092 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6d416-b42c-4f77-b801-14682a3b7226 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:47.732406 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6d416-b42c-4f77-b801-14682a3b7226 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:47.734483 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6d416-b42c-4f77-b801-14682a3b7226 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:47.745418 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d3b6d416-b42c-4f77-b801-14682a3b7226 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:47.894782 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaa4bb2-f9d1-4e2e-a28d-7e29c02e8236 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:47.913136 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353cccf-f5bd-4fb3-970b-d13106aa4d0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:47.916894 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353cccf-f5bd-4fb3-970b-d13106aa4d0f service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:47.929569 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353cccf-f5bd-4fb3-970b-d13106aa4d0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:47.931080 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353cccf-f5bd-4fb3-970b-d13106aa4d0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:47.960867 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1353cccf-f5bd-4fb3-970b-d13106aa4d0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:47.965110 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1353cccf-f5bd-4fb3-970b-d13106aa4d0f service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:48.005254 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 900/1798] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:48.008015 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353cccf-f5bd-4fb3-970b-d13106aa4d0f service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:48.013855 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6d416-b42c-4f77-b801-14682a3b7226 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:48.017506 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6d416-b42c-4f77-b801-14682a3b7226 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:48.032660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353cccf-f5bd-4fb3-970b-d13106aa4d0f service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:48.044740 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f1a6d-e22d-40e8-836c-3fe04aa39088 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:48.045389 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f1a6d-e22d-40e8-836c-3fe04aa39088 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:48.045917 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f1a6d-e22d-40e8-836c-3fe04aa39088 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:48.046198 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f1a6d-e22d-40e8-836c-3fe04aa39088 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:48.046807 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 899/1799] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:47 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 650 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:48.071853 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f1a6d-e22d-40e8-836c-3fe04aa39088 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:48.073793 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f1a6d-e22d-40e8-836c-3fe04aa39088 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:48.102771 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 901/1800] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:49.869033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b46e50-8e21-4964-aafc-d3e08ed18fc9 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:49.869535 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b46e50-8e21-4964-aafc-d3e08ed18fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:49.869535 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b46e50-8e21-4964-aafc-d3e08ed18fc9 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:49.871467 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 900/1801] 10.222.0.45 () {54 vars in 921 bytes} [Mon Aug 1 14:08:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:08:49.892683 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170281e-3a63-4079-be00-f4d4bcbbf1c5 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:49.893245 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170281e-3a63-4079-be00-f4d4bcbbf1c5 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:49.893245 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170281e-3a63-4079-be00-f4d4bcbbf1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:49.957668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-e170281e-3a63-4079-be00-f4d4bcbbf1c5 None None] MFA Rules not processed for user `5b3adc14dcaf4ac79b43bb97fa1ba1d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:49.959098 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e170281e-3a63-4079-be00-f4d4bcbbf1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:50.031888 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:50.354709 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 902/1802] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Aug 1 14:08:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:50.484979 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b46e50-8e21-4964-aafc-d3e08ed18fc9 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:50.486885 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-32b46e50-8e21-4964-aafc-d3e08ed18fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:50.573034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b46e50-8e21-4964-aafc-d3e08ed18fc9 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:50.601423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba6cbc-2eb2-459d-a4fc-8e13991e3075 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:50.601935 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba6cbc-2eb2-459d-a4fc-8e13991e3075 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:50.614948 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba6cbc-2eb2-459d-a4fc-8e13991e3075 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:50.615318 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba6cbc-2eb2-459d-a4fc-8e13991e3075 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:50.642869 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba6cbc-2eb2-459d-a4fc-8e13991e3075 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:50.647280 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba6cbc-2eb2-459d-a4fc-8e13991e3075 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:50.685452 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 901/1803] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:51.317315 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170281e-3a63-4079-be00-f4d4bcbbf1c5 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:51.319052 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e170281e-3a63-4079-be00-f4d4bcbbf1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:51.372781 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170281e-3a63-4079-be00-f4d4bcbbf1c5 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:51.394269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb013c7b-d1a2-40ae-82d5-b74288977d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:51.395675 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb013c7b-d1a2-40ae-82d5-b74288977d2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:51.406862 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb013c7b-d1a2-40ae-82d5-b74288977d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:51.407524 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb013c7b-d1a2-40ae-82d5-b74288977d2c admin admin] PATH_INFO: `/v3/projects/2b493738161d4f9289fc1ea2808c777e` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:51.415306 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb013c7b-d1a2-40ae-82d5-b74288977d2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b493738161d4f9289fc1ea2808c777e)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:51.417776 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb013c7b-d1a2-40ae-82d5-b74288977d2c admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:51.561845 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-fb013c7b-d1a2-40ae-82d5-b74288977d2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b493738161d4f9289fc1ea2808c777e', 'request_id': 'req-fb013c7b-d1a2-40ae-82d5-b74288977d2c'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:51.581286 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-fb013c7b-d1a2-40ae-82d5-b74288977d2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b493738161d4f9289fc1ea2808c777e', 'request_id': 'req-fb013c7b-d1a2-40ae-82d5-b74288977d2c'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:51.607034 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-fb013c7b-d1a2-40ae-82d5-b74288977d2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b493738161d4f9289fc1ea2808c777e', 'request_id': 'req-fb013c7b-d1a2-40ae-82d5-b74288977d2c'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:51.644649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-fb013c7b-d1a2-40ae-82d5-b74288977d2c admin admin] The token cache is being invalidate because project 2b493738161d4f9289fc1ea2808c777e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:08:51.646273 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-fb013c7b-d1a2-40ae-82d5-b74288977d2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:51.647440 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-fb013c7b-d1a2-40ae-82d5-b74288977d2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:51.651334 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 903/1804] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:08:51 2022] DELETE /identity/v3/projects/2b493738161d4f9289fc1ea2808c777e => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:51.656030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba6cbc-2eb2-459d-a4fc-8e13991e3075 service nova] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:51.673387 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba6cbc-2eb2-459d-a4fc-8e13991e3075 service nova] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:51.749168 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb013c7b-d1a2-40ae-82d5-b74288977d2c admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:51.751033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-fb013c7b-d1a2-40ae-82d5-b74288977d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:51.867286 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a1fa4-5b25-4c5a-8229-90ee38c3021a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:51.868041 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a1fa4-5b25-4c5a-8229-90ee38c3021a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:51.868358 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a1fa4-5b25-4c5a-8229-90ee38c3021a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:51.868835 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a1fa4-5b25-4c5a-8229-90ee38c3021a admin admin] PATH_INFO: `/v3/users/3e04c66ab4e44ded9494e9870e6c6732` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:51.870509 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a1fa4-5b25-4c5a-8229-90ee38c3021a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e04c66ab4e44ded9494e9870e6c6732)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:51.879911 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a1fa4-5b25-4c5a-8229-90ee38c3021a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:51.908243 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:51.957431 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb013c7b-d1a2-40ae-82d5-b74288977d2c admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:51.972602 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c148b9e5-2a56-43bd-b018-2ff7e9a82ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:51.973706 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148b9e5-2a56-43bd-b018-2ff7e9a82ee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:51.974258 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148b9e5-2a56-43bd-b018-2ff7e9a82ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:51.974853 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148b9e5-2a56-43bd-b018-2ff7e9a82ee5 admin admin] PATH_INFO: `/v3/projects/392d4b2368d24b35aa90c19524360810` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:51.976698 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c148b9e5-2a56-43bd-b018-2ff7e9a82ee5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=392d4b2368d24b35aa90c19524360810)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:52.003314 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c148b9e5-2a56-43bd-b018-2ff7e9a82ee5 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:52.071105 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e26a1fa4-5b25-4c5a-8229-90ee38c3021a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e04c66ab4e44ded9494e9870e6c6732'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:52.073289 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e26a1fa4-5b25-4c5a-8229-90ee38c3021a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e04c66ab4e44ded9494e9870e6c6732'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:52.133344 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e26a1fa4-5b25-4c5a-8229-90ee38c3021a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e04c66ab4e44ded9494e9870e6c6732'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:52.147199 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e26a1fa4-5b25-4c5a-8229-90ee38c3021a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e04c66ab4e44ded9494e9870e6c6732'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:52.155313 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e26a1fa4-5b25-4c5a-8229-90ee38c3021a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e04c66ab4e44ded9494e9870e6c6732'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:52.189495 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 902/1805] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:08:51 2022] DELETE /identity/v3/users/3e04c66ab4e44ded9494e9870e6c6732 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:52.234691 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:52.243737 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-c148b9e5-2a56-43bd-b018-2ff7e9a82ee5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '392d4b2368d24b35aa90c19524360810', 'request_id': 'req-c148b9e5-2a56-43bd-b018-2ff7e9a82ee5'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:52.255135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-c148b9e5-2a56-43bd-b018-2ff7e9a82ee5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '392d4b2368d24b35aa90c19524360810', 'request_id': 'req-c148b9e5-2a56-43bd-b018-2ff7e9a82ee5'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:52.293198 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-c148b9e5-2a56-43bd-b018-2ff7e9a82ee5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '392d4b2368d24b35aa90c19524360810', 'request_id': 'req-c148b9e5-2a56-43bd-b018-2ff7e9a82ee5'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:52.331478 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-c148b9e5-2a56-43bd-b018-2ff7e9a82ee5 admin admin] The token cache is being invalidate because project 392d4b2368d24b35aa90c19524360810 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:08:52.333016 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-c148b9e5-2a56-43bd-b018-2ff7e9a82ee5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:52.335417 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-c148b9e5-2a56-43bd-b018-2ff7e9a82ee5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:52.345733 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a1fa4-5b25-4c5a-8229-90ee38c3021a admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:52.355336 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e26a1fa4-5b25-4c5a-8229-90ee38c3021a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:52.365375 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 904/1806] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:08:51 2022] DELETE /identity/v3/projects/392d4b2368d24b35aa90c19524360810 => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:52.576852 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a1fa4-5b25-4c5a-8229-90ee38c3021a admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:52.672562 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c148b9e5-2a56-43bd-b018-2ff7e9a82ee5 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:52.674181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c148b9e5-2a56-43bd-b018-2ff7e9a82ee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:52.796229 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e841916-fe68-431e-95aa-fe8dab02667b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:52.799763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e841916-fe68-431e-95aa-fe8dab02667b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:52.800211 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e841916-fe68-431e-95aa-fe8dab02667b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:52.801839 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e841916-fe68-431e-95aa-fe8dab02667b admin admin] PATH_INFO: `/v3/users/9287ab95d897448cb3c7639db31310c0` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:52.806021 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e841916-fe68-431e-95aa-fe8dab02667b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9287ab95d897448cb3c7639db31310c0)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:52.812623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e841916-fe68-431e-95aa-fe8dab02667b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:52.905440 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c148b9e5-2a56-43bd-b018-2ff7e9a82ee5 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:52.921298 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b0c3d-9355-4c1e-90b7-880d51e2d375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:52.922603 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b0c3d-9355-4c1e-90b7-880d51e2d375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:52.923608 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b0c3d-9355-4c1e-90b7-880d51e2d375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:52.924242 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b0c3d-9355-4c1e-90b7-880d51e2d375 admin admin] PATH_INFO: `/v3/users/3f61e3dd7eb1459381df05af41987bea` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:52.930497 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0b0c3d-9355-4c1e-90b7-880d51e2d375 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f61e3dd7eb1459381df05af41987bea)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:52.938831 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0b0c3d-9355-4c1e-90b7-880d51e2d375 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:52.967935 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:52.989605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1e841916-fe68-431e-95aa-fe8dab02667b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9287ab95d897448cb3c7639db31310c0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:52.991826 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1e841916-fe68-431e-95aa-fe8dab02667b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9287ab95d897448cb3c7639db31310c0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:53.021082 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1e841916-fe68-431e-95aa-fe8dab02667b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9287ab95d897448cb3c7639db31310c0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:53.046499 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1e841916-fe68-431e-95aa-fe8dab02667b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9287ab95d897448cb3c7639db31310c0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:53.048215 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1e841916-fe68-431e-95aa-fe8dab02667b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9287ab95d897448cb3c7639db31310c0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:53.071927 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-9d0b0c3d-9355-4c1e-90b7-880d51e2d375 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f61e3dd7eb1459381df05af41987bea'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:53.072784 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-9d0b0c3d-9355-4c1e-90b7-880d51e2d375 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f61e3dd7eb1459381df05af41987bea'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:53.085909 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-9d0b0c3d-9355-4c1e-90b7-880d51e2d375 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f61e3dd7eb1459381df05af41987bea'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:53.087351 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 903/1807] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:08:52 2022] DELETE /identity/v3/users/9287ab95d897448cb3c7639db31310c0 => generated 0 bytes in 742 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:08:53.090477 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6da671-2a5a-4c2b-9580-80b537569ef6 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:53.091612 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6da671-2a5a-4c2b-9580-80b537569ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:53.092351 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6da671-2a5a-4c2b-9580-80b537569ef6 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:53.098060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 904/1808] 10.222.0.45 () {54 vars in 921 bytes} [Mon Aug 1 14:08:53 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:08:53.111580 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-9d0b0c3d-9355-4c1e-90b7-880d51e2d375 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f61e3dd7eb1459381df05af41987bea'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:53.153593 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-9d0b0c3d-9355-4c1e-90b7-880d51e2d375 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f61e3dd7eb1459381df05af41987bea'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:53.164610 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 905/1809] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:08:52 2022] DELETE /identity/v3/users/3f61e3dd7eb1459381df05af41987bea => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:53.167400 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71d3ac-182d-4038-86e9-6bc4937e4eb5 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:53.168063 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71d3ac-182d-4038-86e9-6bc4937e4eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:53.169483 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71d3ac-182d-4038-86e9-6bc4937e4eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:53.184233 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf81ea5-4088-45a3-857a-1d3d81429f0a None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:53.185126 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf81ea5-4088-45a3-857a-1d3d81429f0a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:53.185377 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf81ea5-4088-45a3-857a-1d3d81429f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:53.221343 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-3bf81ea5-4088-45a3-857a-1d3d81429f0a None None] MFA Rules not processed for user `5b3adc14dcaf4ac79b43bb97fa1ba1d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:53.222671 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3bf81ea5-4088-45a3-857a-1d3d81429f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:53.259952 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-aa71d3ac-182d-4038-86e9-6bc4937e4eb5 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:53.267186 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-aa71d3ac-182d-4038-86e9-6bc4937e4eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:53.337663 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:53.418729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 906/1810] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Aug 1 14:08:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:08:53.559562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 905/1811] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:53.735287 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf81ea5-4088-45a3-857a-1d3d81429f0a None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:53.736824 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3bf81ea5-4088-45a3-857a-1d3d81429f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:53.865106 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf81ea5-4088-45a3-857a-1d3d81429f0a None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:53.878058 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf888a12-c6ed-43b6-9588-8e57eed884f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:53.878752 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf888a12-c6ed-43b6-9588-8e57eed884f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:53.879051 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf888a12-c6ed-43b6-9588-8e57eed884f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:53.879429 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf888a12-c6ed-43b6-9588-8e57eed884f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:53.903709 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf888a12-c6ed-43b6-9588-8e57eed884f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:53.906932 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf888a12-c6ed-43b6-9588-8e57eed884f9 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:53.939992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 907/1812] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:08:54.040044 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa71d3ac-182d-4038-86e9-6bc4937e4eb5 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:54.042225 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-aa71d3ac-182d-4038-86e9-6bc4937e4eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:54.095805 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa71d3ac-182d-4038-86e9-6bc4937e4eb5 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:54.125119 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3cd29d-34a6-4bf5-a4d5-b62e1ad4868b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:54.133269 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cd29d-34a6-4bf5-a4d5-b62e1ad4868b service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:54.135967 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cd29d-34a6-4bf5-a4d5-b62e1ad4868b service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:54.136485 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cd29d-34a6-4bf5-a4d5-b62e1ad4868b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:54.177716 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3cd29d-34a6-4bf5-a4d5-b62e1ad4868b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:54.196240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3cd29d-34a6-4bf5-a4d5-b62e1ad4868b service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:54.294380 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 906/1813] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:54.707080 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf888a12-c6ed-43b6-9588-8e57eed884f9 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:54.709995 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-bf888a12-c6ed-43b6-9588-8e57eed884f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:54.743046 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf888a12-c6ed-43b6-9588-8e57eed884f9 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:54.768891 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba1cf2-9e3a-4fbd-8d38-746e3ee15240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:54.769635 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba1cf2-9e3a-4fbd-8d38-746e3ee15240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:54.769918 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba1cf2-9e3a-4fbd-8d38-746e3ee15240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:54.770372 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba1cf2-9e3a-4fbd-8d38-746e3ee15240 admin admin] PATH_INFO: `/v3/projects/ec2cb6858b8f450aa72fc275d9c882fe` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:54.773482 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba1cf2-9e3a-4fbd-8d38-746e3ee15240 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec2cb6858b8f450aa72fc275d9c882fe)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:54.776615 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba1cf2-9e3a-4fbd-8d38-746e3ee15240 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:54.918118 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-efba1cf2-9e3a-4fbd-8d38-746e3ee15240 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec2cb6858b8f450aa72fc275d9c882fe', 'request_id': 'req-efba1cf2-9e3a-4fbd-8d38-746e3ee15240'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:54.925705 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-efba1cf2-9e3a-4fbd-8d38-746e3ee15240 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec2cb6858b8f450aa72fc275d9c882fe', 'request_id': 'req-efba1cf2-9e3a-4fbd-8d38-746e3ee15240'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:54.937212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-efba1cf2-9e3a-4fbd-8d38-746e3ee15240 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec2cb6858b8f450aa72fc275d9c882fe', 'request_id': 'req-efba1cf2-9e3a-4fbd-8d38-746e3ee15240'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:54.953860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-efba1cf2-9e3a-4fbd-8d38-746e3ee15240 admin admin] The token cache is being invalidate because project ec2cb6858b8f450aa72fc275d9c882fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:08:54.954281 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-efba1cf2-9e3a-4fbd-8d38-746e3ee15240 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:54.955952 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-efba1cf2-9e3a-4fbd-8d38-746e3ee15240 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:54.964232 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 908/1814] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:08:54 2022] DELETE /identity/v3/projects/ec2cb6858b8f450aa72fc275d9c882fe => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:54.964457 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef9a0b-d09b-42ca-ba39-5510887efa05 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:54.964635 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef9a0b-d09b-42ca-ba39-5510887efa05 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:54.965193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef9a0b-d09b-42ca-ba39-5510887efa05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:55.003941 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-01ef9a0b-d09b-42ca-ba39-5510887efa05 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:55.006824 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-01ef9a0b-d09b-42ca-ba39-5510887efa05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:55.128989 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 907/1815] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:55.136804 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052d49f-cedb-4ae9-9da4-bb0a5a42fcb6 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:55.137153 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052d49f-cedb-4ae9-9da4-bb0a5a42fcb6 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:55.137430 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052d49f-cedb-4ae9-9da4-bb0a5a42fcb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:55.161082 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-4052d49f-cedb-4ae9-9da4-bb0a5a42fcb6 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:55.162324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-4052d49f-cedb-4ae9-9da4-bb0a5a42fcb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:55.177537 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 909/1816] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:55.182348 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef9a0b-d09b-42ca-ba39-5510887efa05 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:55.196420 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4052d49f-cedb-4ae9-9da4-bb0a5a42fcb6 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:55.200792 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-4052d49f-cedb-4ae9-9da4-bb0a5a42fcb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:55.202297 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef9a0b-d09b-42ca-ba39-5510887efa05 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:55.211782 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e43cd-6c9c-4bc3-8bac-5c675dc3ec10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:55.214162 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e43cd-6c9c-4bc3-8bac-5c675dc3ec10 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:55.214772 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e43cd-6c9c-4bc3-8bac-5c675dc3ec10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:55.215366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e43cd-6c9c-4bc3-8bac-5c675dc3ec10 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:55.216202 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5e43cd-6c9c-4bc3-8bac-5c675dc3ec10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:55.216618 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5e43cd-6c9c-4bc3-8bac-5c675dc3ec10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:55.217368 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4052d49f-cedb-4ae9-9da4-bb0a5a42fcb6 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:55.222110 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5e43cd-6c9c-4bc3-8bac-5c675dc3ec10 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:55.226396 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f712f2-1fda-41a7-b4ce-d065cb66848f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:55.227323 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f712f2-1fda-41a7-b4ce-d065cb66848f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:55.227772 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f712f2-1fda-41a7-b4ce-d065cb66848f admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:55.228219 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f712f2-1fda-41a7-b4ce-d065cb66848f admin admin] PATH_INFO: `/v3/users/e14cb0cb0b364747bc3e812a79a88911` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:55.231454 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f712f2-1fda-41a7-b4ce-d065cb66848f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e14cb0cb0b364747bc3e812a79a88911)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:55.236387 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f712f2-1fda-41a7-b4ce-d065cb66848f admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:55.247336 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 908/1817] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:08:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:55.248340 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e43cd-6c9c-4bc3-8bac-5c675dc3ec10 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:55.251245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6f5e43cd-6c9c-4bc3-8bac-5c675dc3ec10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:55.271733 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e43cd-6c9c-4bc3-8bac-5c675dc3ec10 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:55.278283 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e411c89-21b1-47cf-806c-a83c2ebd0d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:55.280226 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e411c89-21b1-47cf-806c-a83c2ebd0d67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:55.280400 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e411c89-21b1-47cf-806c-a83c2ebd0d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:55.280665 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e411c89-21b1-47cf-806c-a83c2ebd0d67 admin admin] PATH_INFO: `/v3/projects/1d5300c73af44847a15c9f459975b59d` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:55.283316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e411c89-21b1-47cf-806c-a83c2ebd0d67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d5300c73af44847a15c9f459975b59d)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:55.285361 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e411c89-21b1-47cf-806c-a83c2ebd0d67 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:55.320100 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:55.349245 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-34f712f2-1fda-41a7-b4ce-d065cb66848f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e14cb0cb0b364747bc3e812a79a88911'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:55.350176 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-34f712f2-1fda-41a7-b4ce-d065cb66848f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e14cb0cb0b364747bc3e812a79a88911'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:55.364642 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6e411c89-21b1-47cf-806c-a83c2ebd0d67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d5300c73af44847a15c9f459975b59d', 'request_id': 'req-6e411c89-21b1-47cf-806c-a83c2ebd0d67'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:55.366793 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-34f712f2-1fda-41a7-b4ce-d065cb66848f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e14cb0cb0b364747bc3e812a79a88911'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:55.372792 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:55.373070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6e411c89-21b1-47cf-806c-a83c2ebd0d67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d5300c73af44847a15c9f459975b59d', 'request_id': 'req-6e411c89-21b1-47cf-806c-a83c2ebd0d67'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:55.384931 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6e411c89-21b1-47cf-806c-a83c2ebd0d67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d5300c73af44847a15c9f459975b59d', 'request_id': 'req-6e411c89-21b1-47cf-806c-a83c2ebd0d67'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:55.387528 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-34f712f2-1fda-41a7-b4ce-d065cb66848f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e14cb0cb0b364747bc3e812a79a88911'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:55.404723 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6e411c89-21b1-47cf-806c-a83c2ebd0d67 admin admin] The token cache is being invalidate because project 1d5300c73af44847a15c9f459975b59d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:08:55.405031 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6e411c89-21b1-47cf-806c-a83c2ebd0d67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:55.407468 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6e411c89-21b1-47cf-806c-a83c2ebd0d67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:55.416541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 909/1818] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:08:55 2022] DELETE /identity/v3/projects/1d5300c73af44847a15c9f459975b59d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:55.416541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e411c89-21b1-47cf-806c-a83c2ebd0d67 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:55.416541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6e411c89-21b1-47cf-806c-a83c2ebd0d67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:55.421170 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-34f712f2-1fda-41a7-b4ce-d065cb66848f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e14cb0cb0b364747bc3e812a79a88911'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:55.435962 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 910/1819] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:08:55 2022] DELETE /identity/v3/users/e14cb0cb0b364747bc3e812a79a88911 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:55.437566 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641f7aa-383f-4412-86cb-027630cd1a5f None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:55.437960 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641f7aa-383f-4412-86cb-027630cd1a5f None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:55.438347 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641f7aa-383f-4412-86cb-027630cd1a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:55.487572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-e641f7aa-383f-4412-86cb-027630cd1a5f None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:55.489523 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e641f7aa-383f-4412-86cb-027630cd1a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:55.567255 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e411c89-21b1-47cf-806c-a83c2ebd0d67 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:55.669167 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 911/1820] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:55.676572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209a616-d358-4c74-88a0-cd0a6d260594 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:55.676853 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209a616-d358-4c74-88a0-cd0a6d260594 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:55.677128 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209a616-d358-4c74-88a0-cd0a6d260594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:55.693674 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a36918-7e59-475f-9115-4b923e3a5e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:55.694475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a36918-7e59-475f-9115-4b923e3a5e3b admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:55.694820 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a36918-7e59-475f-9115-4b923e3a5e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:55.695141 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a36918-7e59-475f-9115-4b923e3a5e3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:55.695879 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a36918-7e59-475f-9115-4b923e3a5e3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:55.698597 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a36918-7e59-475f-9115-4b923e3a5e3b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:55.706233 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-1209a616-d358-4c74-88a0-cd0a6d260594 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:55.707632 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1209a616-d358-4c74-88a0-cd0a6d260594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:55.718746 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 912/1821] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:55.719645 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209a616-d358-4c74-88a0-cd0a6d260594 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:55.721478 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1209a616-d358-4c74-88a0-cd0a6d260594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:55.735807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209a616-d358-4c74-88a0-cd0a6d260594 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:55.736136 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 910/1822] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:55 2022] POST /identity/v3/projects => generated 409 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:55.737654 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50afc27f-c4f1-42a4-9362-e85ccaaa01d2 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:55.738192 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50afc27f-c4f1-42a4-9362-e85ccaaa01d2 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:55.738473 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50afc27f-c4f1-42a4-9362-e85ccaaa01d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:55.765620 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-50afc27f-c4f1-42a4-9362-e85ccaaa01d2 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:55.766942 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-50afc27f-c4f1-42a4-9362-e85ccaaa01d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:55.807773 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ec6fa-b8a5-4b81-9dcf-f190157e9465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:55.808574 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ec6fa-b8a5-4b81-9dcf-f190157e9465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:55.808973 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ec6fa-b8a5-4b81-9dcf-f190157e9465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:55.809130 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ec6fa-b8a5-4b81-9dcf-f190157e9465 admin admin] PATH_INFO: `/v3/projects/6c620caee24342328e7bd4d44cbec2de` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:55.810580 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850ec6fa-b8a5-4b81-9dcf-f190157e9465 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c620caee24342328e7bd4d44cbec2de)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:55.812555 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850ec6fa-b8a5-4b81-9dcf-f190157e9465 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:55.815701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:55.841258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 911/1823] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:55.842285 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50afc27f-c4f1-42a4-9362-e85ccaaa01d2 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:55.843784 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-50afc27f-c4f1-42a4-9362-e85ccaaa01d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:55.873350 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-850ec6fa-b8a5-4b81-9dcf-f190157e9465 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c620caee24342328e7bd4d44cbec2de', 'request_id': 'req-850ec6fa-b8a5-4b81-9dcf-f190157e9465'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:55.880088 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-850ec6fa-b8a5-4b81-9dcf-f190157e9465 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c620caee24342328e7bd4d44cbec2de', 'request_id': 'req-850ec6fa-b8a5-4b81-9dcf-f190157e9465'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:55.889374 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-850ec6fa-b8a5-4b81-9dcf-f190157e9465 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c620caee24342328e7bd4d44cbec2de', 'request_id': 'req-850ec6fa-b8a5-4b81-9dcf-f190157e9465'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:55.897410 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:55.902501 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-850ec6fa-b8a5-4b81-9dcf-f190157e9465 admin admin] The token cache is being invalidate because project 6c620caee24342328e7bd4d44cbec2de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:08:55.903539 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-850ec6fa-b8a5-4b81-9dcf-f190157e9465 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:55.904853 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-850ec6fa-b8a5-4b81-9dcf-f190157e9465 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:55.907100 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 913/1824] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:08:55 2022] DELETE /identity/v3/projects/6c620caee24342328e7bd4d44cbec2de => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:55.909404 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50afc27f-c4f1-42a4-9362-e85ccaaa01d2 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:55.909954 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ec6fa-b8a5-4b81-9dcf-f190157e9465 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:55.912026 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-850ec6fa-b8a5-4b81-9dcf-f190157e9465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:55.971920 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539efd6-3ab0-4ec1-bc4f-5a0457ce6516 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:55.972585 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539efd6-3ab0-4ec1-bc4f-5a0457ce6516 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:55.972848 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539efd6-3ab0-4ec1-bc4f-5a0457ce6516 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:55.973115 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539efd6-3ab0-4ec1-bc4f-5a0457ce6516 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:55.975938 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d539efd6-3ab0-4ec1-bc4f-5a0457ce6516 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:55.990491 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539efd6-3ab0-4ec1-bc4f-5a0457ce6516 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:55.990876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ec6fa-b8a5-4b81-9dcf-f190157e9465 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:55.992233 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539efd6-3ab0-4ec1-bc4f-5a0457ce6516 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:55.996337 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0002e270-1997-4040-b2c9-c33e2a558260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:55.996714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002e270-1997-4040-b2c9-c33e2a558260 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:55.997110 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002e270-1997-4040-b2c9-c33e2a558260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:55.997239 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002e270-1997-4040-b2c9-c33e2a558260 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:55.998754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0002e270-1997-4040-b2c9-c33e2a558260 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:55.999872 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0002e270-1997-4040-b2c9-c33e2a558260 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:56.004352 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 912/1825] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:56.005211 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539efd6-3ab0-4ec1-bc4f-5a0457ce6516 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:56.007119 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d539efd6-3ab0-4ec1-bc4f-5a0457ce6516 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:56.020074 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539efd6-3ab0-4ec1-bc4f-5a0457ce6516 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:56.025813 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9042d54-9bf3-4c01-8a11-31527564fd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:56.026457 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9042d54-9bf3-4c01-8a11-31527564fd1f admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:56.026770 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 914/1826] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:55 2022] POST /identity/v3/users => generated 305 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:56.027074 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9042d54-9bf3-4c01-8a11-31527564fd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:56.027404 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9042d54-9bf3-4c01-8a11-31527564fd1f admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:56.028550 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9042d54-9bf3-4c01-8a11-31527564fd1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:56.028851 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9042d54-9bf3-4c01-8a11-31527564fd1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:56.031202 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9042d54-9bf3-4c01-8a11-31527564fd1f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:56.031608 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7f01e-cea3-4407-b4ad-7fcc4da6cdf2 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:56.031901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7f01e-cea3-4407-b4ad-7fcc4da6cdf2 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:56.032176 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7f01e-cea3-4407-b4ad-7fcc4da6cdf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:56.048705 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 913/1827] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:08:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:56.048705 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9042d54-9bf3-4c01-8a11-31527564fd1f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:56.054951 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-7fe7f01e-cea3-4407-b4ad-7fcc4da6cdf2 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:56.056173 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7fe7f01e-cea3-4407-b4ad-7fcc4da6cdf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:56.062905 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9042d54-9bf3-4c01-8a11-31527564fd1f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:56.066183 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 915/1828] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:56.068977 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda2efb6-b993-4d9f-b7eb-04ad99c4dcd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:56.069630 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2efb6-b993-4d9f-b7eb-04ad99c4dcd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:56.069915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2efb6-b993-4d9f-b7eb-04ad99c4dcd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:56.070196 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2efb6-b993-4d9f-b7eb-04ad99c4dcd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:56.071427 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda2efb6-b993-4d9f-b7eb-04ad99c4dcd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:56.072583 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda2efb6-b993-4d9f-b7eb-04ad99c4dcd1 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:56.075767 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7f01e-cea3-4407-b4ad-7fcc4da6cdf2 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:56.086836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 914/1829] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:56.088117 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3fa11-b9b2-4ff3-bb52-9b3d9f4d5703 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:56.088438 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3fa11-b9b2-4ff3-bb52-9b3d9f4d5703 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:56.088716 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3fa11-b9b2-4ff3-bb52-9b3d9f4d5703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:56.089134 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7f01e-cea3-4407-b4ad-7fcc4da6cdf2 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:56.094695 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c534c7-e8e6-4d08-98a5-adc952345e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:56.095331 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c534c7-e8e6-4d08-98a5-adc952345e57 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:56.095841 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c534c7-e8e6-4d08-98a5-adc952345e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:56.095989 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c534c7-e8e6-4d08-98a5-adc952345e57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:56.096786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c534c7-e8e6-4d08-98a5-adc952345e57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:56.098834 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c534c7-e8e6-4d08-98a5-adc952345e57 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:56.112590 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-91c3fa11-b9b2-4ff3-bb52-9b3d9f4d5703 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:56.114130 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-91c3fa11-b9b2-4ff3-bb52-9b3d9f4d5703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:56.125408 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 915/1830] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:56.126889 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c3fa11-b9b2-4ff3-bb52-9b3d9f4d5703 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:56.137409 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 916/1831] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:56 2022] POST /identity/v3/projects => generated 379 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:56.138198 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c534c7-e8e6-4d08-98a5-adc952345e57 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:56.140685 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c3fa11-b9b2-4ff3-bb52-9b3d9f4d5703 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:56.215708 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b99635-c187-481b-a5eb-2be424605b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:56.216568 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b99635-c187-481b-a5eb-2be424605b24 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:56.216568 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b99635-c187-481b-a5eb-2be424605b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:56.216853 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b99635-c187-481b-a5eb-2be424605b24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:56.217677 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b99635-c187-481b-a5eb-2be424605b24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:56.219284 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b99635-c187-481b-a5eb-2be424605b24 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:56.220485 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c534c7-e8e6-4d08-98a5-adc952345e57 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:56.227652 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f72ff9-dd13-4e50-b9f1-561615d9c4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:56.228517 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f72ff9-dd13-4e50-b9f1-561615d9c4c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:56.228805 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f72ff9-dd13-4e50-b9f1-561615d9c4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:56.229085 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f72ff9-dd13-4e50-b9f1-561615d9c4c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:56.229805 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f72ff9-dd13-4e50-b9f1-561615d9c4c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:56.230117 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f72ff9-dd13-4e50-b9f1-561615d9c4c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:56.233563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 916/1832] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:56.233563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b99635-c187-481b-a5eb-2be424605b24 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:56.233803 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f72ff9-dd13-4e50-b9f1-561615d9c4c6 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:56.241788 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b99635-c187-481b-a5eb-2be424605b24 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:56.247580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8f762-3a3b-46ec-b4a3-5ed59048a217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:56.248463 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8f762-3a3b-46ec-b4a3-5ed59048a217 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:56.248752 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 917/1833] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:08:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:56.248985 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8f762-3a3b-46ec-b4a3-5ed59048a217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:56.249265 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8f762-3a3b-46ec-b4a3-5ed59048a217 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:56.249950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f72ff9-dd13-4e50-b9f1-561615d9c4c6 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:56.250057 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d8f762-3a3b-46ec-b4a3-5ed59048a217 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:56.251885 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d8f762-3a3b-46ec-b4a3-5ed59048a217 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:56.259860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f72ff9-dd13-4e50-b9f1-561615d9c4c6 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:56.261652 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:56.266631 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0ba4c-db51-4f1c-8ada-43c8a12a9f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:56.267629 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0ba4c-db51-4f1c-8ada-43c8a12a9f5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:56.267985 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0ba4c-db51-4f1c-8ada-43c8a12a9f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:56.268143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0ba4c-db51-4f1c-8ada-43c8a12a9f5d admin admin] PATH_INFO: `/v3/projects/9b6a56cd8c4a4b73bfbccb3d5f7f0ba4/users/b5297b694c9b414ab5947c5545f07596/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:56.281483 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 917/1834] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:56 2022] POST /identity/v3/users => generated 290 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:56.282553 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8f762-3a3b-46ec-b4a3-5ed59048a217 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:56.289938 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c0ba4c-db51-4f1c-8ada-43c8a12a9f5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b6a56cd8c4a4b73bfbccb3d5f7f0ba4, user_id=b5297b694c9b414ab5947c5545f07596, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:56.293102 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c0ba4c-db51-4f1c-8ada-43c8a12a9f5d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:56.294501 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8f762-3a3b-46ec-b4a3-5ed59048a217 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:56.299501 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb00dc0-39c2-4df3-b452-a3c41cc5fa4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:56.300201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb00dc0-39c2-4df3-b452-a3c41cc5fa4b admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:56.300540 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb00dc0-39c2-4df3-b452-a3c41cc5fa4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:56.300812 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb00dc0-39c2-4df3-b452-a3c41cc5fa4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:56.301497 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb00dc0-39c2-4df3-b452-a3c41cc5fa4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:56.303430 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb00dc0-39c2-4df3-b452-a3c41cc5fa4b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:56.310958 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 918/1835] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:56 2022] PUT /identity/v3/projects/9b6a56cd8c4a4b73bfbccb3d5f7f0ba4/users/b5297b694c9b414ab5947c5545f07596/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:56.311828 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0ba4c-db51-4f1c-8ada-43c8a12a9f5d admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:56.340318 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 918/1836] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:56 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:56.341857 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10af227-53a6-4b5c-bbed-30e9a0094852 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:56.342410 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10af227-53a6-4b5c-bbed-30e9a0094852 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:56.342629 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10af227-53a6-4b5c-bbed-30e9a0094852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:56.362447 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:56.379056 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a10af227-53a6-4b5c-bbed-30e9a0094852 None None] MFA Rules not processed for user `b5297b694c9b414ab5947c5545f07596`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:56.380340 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a10af227-53a6-4b5c-bbed-30e9a0094852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:56.398286 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0ba4c-db51-4f1c-8ada-43c8a12a9f5d admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:56.453903 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 919/1837] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:56 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:56.454952 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10af227-53a6-4b5c-bbed-30e9a0094852 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:56.456332 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a10af227-53a6-4b5c-bbed-30e9a0094852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:56.463648 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3a84b-bf3a-4fb7-8e93-cb28a23c75df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:56.464295 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3a84b-bf3a-4fb7-8e93-cb28a23c75df admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:56.464605 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3a84b-bf3a-4fb7-8e93-cb28a23c75df admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:56.464907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3a84b-bf3a-4fb7-8e93-cb28a23c75df admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:56.465602 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac3a84b-bf3a-4fb7-8e93-cb28a23c75df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:56.467533 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac3a84b-bf3a-4fb7-8e93-cb28a23c75df admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:56.482167 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 919/1838] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:56.483083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3a84b-bf3a-4fb7-8e93-cb28a23c75df admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:56.493960 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3a84b-bf3a-4fb7-8e93-cb28a23c75df admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:56.502341 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee86328-ccc4-40a0-ac98-7aa8bb6a45c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:56.503187 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee86328-ccc4-40a0-ac98-7aa8bb6a45c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:56.503468 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee86328-ccc4-40a0-ac98-7aa8bb6a45c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:56.503744 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee86328-ccc4-40a0-ac98-7aa8bb6a45c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:56.504453 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee86328-ccc4-40a0-ac98-7aa8bb6a45c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:56.506441 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee86328-ccc4-40a0-ac98-7aa8bb6a45c4 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:56.539139 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10af227-53a6-4b5c-bbed-30e9a0094852 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:56.544560 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebe7c5e-2eb8-455b-81d1-c9ca69c017ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:56.545233 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe7c5e-2eb8-455b-81d1-c9ca69c017ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:56.545532 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe7c5e-2eb8-455b-81d1-c9ca69c017ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:56.545807 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe7c5e-2eb8-455b-81d1-c9ca69c017ad admin admin] PATH_INFO: `/v3/projects/8ee6ba162508458fb76074fb13f161ca` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:56.547406 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebe7c5e-2eb8-455b-81d1-c9ca69c017ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ee6ba162508458fb76074fb13f161ca)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:56.549830 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebe7c5e-2eb8-455b-81d1-c9ca69c017ad admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:56.557614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 920/1839] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:56 2022] POST /identity/v3/users => generated 286 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:56.559090 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee86328-ccc4-40a0-ac98-7aa8bb6a45c4 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:56.593776 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee86328-ccc4-40a0-ac98-7aa8bb6a45c4 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:56.606870 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a195d5f-a53d-4d1a-83e1-6773d70cb141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:56.607710 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a195d5f-a53d-4d1a-83e1-6773d70cb141 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:56.607819 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a195d5f-a53d-4d1a-83e1-6773d70cb141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:56.608052 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a195d5f-a53d-4d1a-83e1-6773d70cb141 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:56.610921 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a195d5f-a53d-4d1a-83e1-6773d70cb141 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:56.616915 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a195d5f-a53d-4d1a-83e1-6773d70cb141 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:56.628599 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-eebe7c5e-2eb8-455b-81d1-c9ca69c017ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ee6ba162508458fb76074fb13f161ca', 'request_id': 'req-eebe7c5e-2eb8-455b-81d1-c9ca69c017ad'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:56.635477 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-eebe7c5e-2eb8-455b-81d1-c9ca69c017ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ee6ba162508458fb76074fb13f161ca', 'request_id': 'req-eebe7c5e-2eb8-455b-81d1-c9ca69c017ad'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:56.637807 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:INSERT=4,keystone:SELECT=90,keystone:DELETE=6 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:56.649981 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-eebe7c5e-2eb8-455b-81d1-c9ca69c017ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ee6ba162508458fb76074fb13f161ca', 'request_id': 'req-eebe7c5e-2eb8-455b-81d1-c9ca69c017ad'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:56.663370 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 921/1840] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:56 2022] POST /identity/v3/projects => generated 409 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:56.665375 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a195d5f-a53d-4d1a-83e1-6773d70cb141 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:56.678298 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-eebe7c5e-2eb8-455b-81d1-c9ca69c017ad admin admin] The token cache is being invalidate because project 8ee6ba162508458fb76074fb13f161ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:08:56.678624 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-eebe7c5e-2eb8-455b-81d1-c9ca69c017ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:56.679508 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-eebe7c5e-2eb8-455b-81d1-c9ca69c017ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:56.683914 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 920/1841] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:08:56 2022] DELETE /identity/v3/projects/8ee6ba162508458fb76074fb13f161ca => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:56.683914 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebe7c5e-2eb8-455b-81d1-c9ca69c017ad admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:56.685980 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-eebe7c5e-2eb8-455b-81d1-c9ca69c017ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:56.753121 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a195d5f-a53d-4d1a-83e1-6773d70cb141 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:56.758234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19694cf3-975b-4ed2-ad4b-4ac1697a9410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:56.758948 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19694cf3-975b-4ed2-ad4b-4ac1697a9410 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:56.759458 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19694cf3-975b-4ed2-ad4b-4ac1697a9410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:56.759807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19694cf3-975b-4ed2-ad4b-4ac1697a9410 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:56.761061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19694cf3-975b-4ed2-ad4b-4ac1697a9410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:56.764029 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19694cf3-975b-4ed2-ad4b-4ac1697a9410 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:56.783136 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 922/1842] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:56.784941 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19694cf3-975b-4ed2-ad4b-4ac1697a9410 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:56.787962 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-19694cf3-975b-4ed2-ad4b-4ac1697a9410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:56.812212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19694cf3-975b-4ed2-ad4b-4ac1697a9410 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:56.818850 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebe7c5e-2eb8-455b-81d1-c9ca69c017ad admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:56.822787 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a227b783-b1f6-4a06-b93d-e7db35a62032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:56.824258 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a227b783-b1f6-4a06-b93d-e7db35a62032 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:56.825789 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c3d0e-da00-44ef-8e88-64a499934a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:56.826649 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c3d0e-da00-44ef-8e88-64a499934a7c admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:56.827254 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c3d0e-da00-44ef-8e88-64a499934a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:56.828239 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c3d0e-da00-44ef-8e88-64a499934a7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:56.829414 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c3d0e-da00-44ef-8e88-64a499934a7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:56.831352 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c3d0e-da00-44ef-8e88-64a499934a7c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:56.836565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a227b783-b1f6-4a06-b93d-e7db35a62032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:56.837037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a227b783-b1f6-4a06-b93d-e7db35a62032 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:56.838048 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a227b783-b1f6-4a06-b93d-e7db35a62032 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:56.841249 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a227b783-b1f6-4a06-b93d-e7db35a62032 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:56.845496 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 921/1843] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:56.846660 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c3d0e-da00-44ef-8e88-64a499934a7c admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:56.848978 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f92c3d0e-da00-44ef-8e88-64a499934a7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:56.868960 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c3d0e-da00-44ef-8e88-64a499934a7c admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:56.877326 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c73dc-19c2-4866-918a-96becc0758df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:56.878675 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c73dc-19c2-4866-918a-96becc0758df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:56.879082 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c73dc-19c2-4866-918a-96becc0758df admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:56.879482 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c73dc-19c2-4866-918a-96becc0758df admin admin] PATH_INFO: `/v3/users/5c9209d97cc644049618043b70ab1cba` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:56.882134 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80c73dc-19c2-4866-918a-96becc0758df admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c9209d97cc644049618043b70ab1cba)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:56.882429 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 923/1844] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:56 2022] POST /identity/v3/users => generated 311 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:56.883423 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a227b783-b1f6-4a06-b93d-e7db35a62032 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:56.887507 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80c73dc-19c2-4866-918a-96becc0758df admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:56.888944 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a227b783-b1f6-4a06-b93d-e7db35a62032 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:56.904385 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a227b783-b1f6-4a06-b93d-e7db35a62032 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:56.909698 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd997d-c9eb-4496-8ac7-ded820a65b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:56.912303 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd997d-c9eb-4496-8ac7-ded820a65b47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:56.912303 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd997d-c9eb-4496-8ac7-ded820a65b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:56.912303 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd997d-c9eb-4496-8ac7-ded820a65b47 admin admin] PATH_INFO: `/v3/projects/8a6a587aedea4024aa5cf14dbe3f51e5/users/1423bab4b1cd4ac5acc7ebe55b1f27ef/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:56.921822 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:56.934694 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd997d-c9eb-4496-8ac7-ded820a65b47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a6a587aedea4024aa5cf14dbe3f51e5, user_id=1423bab4b1cd4ac5acc7ebe55b1f27ef, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:56.938818 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd997d-c9eb-4496-8ac7-ded820a65b47 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:56.972742 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 924/1845] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:56 2022] PUT /identity/v3/projects/8a6a587aedea4024aa5cf14dbe3f51e5/users/1423bab4b1cd4ac5acc7ebe55b1f27ef/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:56.973809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd997d-c9eb-4496-8ac7-ded820a65b47 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:56.999565 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d80c73dc-19c2-4866-918a-96becc0758df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c9209d97cc644049618043b70ab1cba'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:57.000372 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d80c73dc-19c2-4866-918a-96becc0758df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c9209d97cc644049618043b70ab1cba'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:57.016180 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d80c73dc-19c2-4866-918a-96becc0758df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c9209d97cc644049618043b70ab1cba'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:57.024006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d80c73dc-19c2-4866-918a-96becc0758df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c9209d97cc644049618043b70ab1cba'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:57.024922 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d80c73dc-19c2-4866-918a-96becc0758df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c9209d97cc644049618043b70ab1cba'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:57.043476 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 922/1846] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:08:56 2022] DELETE /identity/v3/users/5c9209d97cc644049618043b70ab1cba => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:57.046691 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c73dc-19c2-4866-918a-96becc0758df admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:57.046972 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d80c73dc-19c2-4866-918a-96becc0758df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:57.082936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd997d-c9eb-4496-8ac7-ded820a65b47 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:57.135847 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:57.142483 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c73dc-19c2-4866-918a-96becc0758df admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:57.151339 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f5a99-ce02-4409-850b-673c883d5ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:57.151973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f5a99-ce02-4409-850b-673c883d5ffc admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:57.152258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f5a99-ce02-4409-850b-673c883d5ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:57.152631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f5a99-ce02-4409-850b-673c883d5ffc admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:57.153748 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9f5a99-ce02-4409-850b-673c883d5ffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:57.155413 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9f5a99-ce02-4409-850b-673c883d5ffc admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:57.167322 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ace35-905c-4217-a4d8-b80d3ba3b4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:57.167479 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 923/1847] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:57.168056 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ace35-905c-4217-a4d8-b80d3ba3b4f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:57.168380 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ace35-905c-4217-a4d8-b80d3ba3b4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:57.168657 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ace35-905c-4217-a4d8-b80d3ba3b4f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:57.169105 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59a294-a32c-4fb4-a580-5f2e1723b883 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:57.169376 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ace35-905c-4217-a4d8-b80d3ba3b4f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:57.169567 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59a294-a32c-4fb4-a580-5f2e1723b883 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:57.169955 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59a294-a32c-4fb4-a580-5f2e1723b883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:57.171962 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ace35-905c-4217-a4d8-b80d3ba3b4f6 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:57.186413 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 925/1848] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:57.189823 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ace35-905c-4217-a4d8-b80d3ba3b4f6 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:57.202268 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ace35-905c-4217-a4d8-b80d3ba3b4f6 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:57.202642 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-be59a294-a32c-4fb4-a580-5f2e1723b883 None None] MFA Rules not processed for user `1423bab4b1cd4ac5acc7ebe55b1f27ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:57.204563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-be59a294-a32c-4fb4-a580-5f2e1723b883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:57.207334 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05082bfe-f5e8-4f36-89cc-5c5d6038616c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:57.208066 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05082bfe-f5e8-4f36-89cc-5c5d6038616c admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:57.208418 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05082bfe-f5e8-4f36-89cc-5c5d6038616c admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:57.208702 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05082bfe-f5e8-4f36-89cc-5c5d6038616c admin admin] PATH_INFO: `/v3/projects/3d66b31a768742318b4de6e0b3d74c4a/users/4e8e9f312c104af39cb20091fa4024ad/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:57.234237 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05082bfe-f5e8-4f36-89cc-5c5d6038616c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d66b31a768742318b4de6e0b3d74c4a, user_id=4e8e9f312c104af39cb20091fa4024ad, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:57.237593 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05082bfe-f5e8-4f36-89cc-5c5d6038616c admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:57.255206 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 926/1849] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:57 2022] PUT /identity/v3/projects/3d66b31a768742318b4de6e0b3d74c4a/users/4e8e9f312c104af39cb20091fa4024ad/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:57.255944 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05082bfe-f5e8-4f36-89cc-5c5d6038616c admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:57.258536 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-05082bfe-f5e8-4f36-89cc-5c5d6038616c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:57.291825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 924/1850] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:57.292683 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59a294-a32c-4fb4-a580-5f2e1723b883 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:57.305082 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59a294-a32c-4fb4-a580-5f2e1723b883 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:57.333950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05082bfe-f5e8-4f36-89cc-5c5d6038616c admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:57.340245 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b732e99-014e-4113-ab85-ee3c4d6b0a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:57.341134 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b732e99-014e-4113-ab85-ee3c4d6b0a4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:57.346815 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b732e99-014e-4113-ab85-ee3c4d6b0a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:57.350918 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b732e99-014e-4113-ab85-ee3c4d6b0a4c admin admin] PATH_INFO: `/v3/projects/ac97d680a31b42aa978d067b7af07d24/users/d4f68089787b4e7f8fbbb65ecfafcec0/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:57.391608 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:57.393500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b732e99-014e-4113-ab85-ee3c4d6b0a4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac97d680a31b42aa978d067b7af07d24, user_id=d4f68089787b4e7f8fbbb65ecfafcec0, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:57.396862 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ec221-a11f-486c-b264-d75c9df97157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:57.397116 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b732e99-014e-4113-ab85-ee3c4d6b0a4c admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:57.398069 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ec221-a11f-486c-b264-d75c9df97157 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:57.401126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ec221-a11f-486c-b264-d75c9df97157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:57.401676 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ec221-a11f-486c-b264-d75c9df97157 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:57.403139 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ec221-a11f-486c-b264-d75c9df97157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:57.407000 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ec221-a11f-486c-b264-d75c9df97157 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:57.425341 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 927/1851] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:57 2022] PUT /identity/v3/projects/ac97d680a31b42aa978d067b7af07d24/users/d4f68089787b4e7f8fbbb65ecfafcec0/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:57.425341 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b732e99-014e-4113-ab85-ee3c4d6b0a4c admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:57.431604 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-8b732e99-014e-4113-ab85-ee3c4d6b0a4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:57.445208 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 925/1852] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:57.446185 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ec221-a11f-486c-b264-d75c9df97157 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:57.460327 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-cc1ec221-a11f-486c-b264-d75c9df97157 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:57.484629 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ec221-a11f-486c-b264-d75c9df97157 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:57.548816 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b732e99-014e-4113-ab85-ee3c4d6b0a4c admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:57.559602 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c09c3-3b70-4f31-8446-0669c0db71f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:57.560428 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c09c3-3b70-4f31-8446-0669c0db71f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:57.560745 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c09c3-3b70-4f31-8446-0669c0db71f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:57.561022 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c09c3-3b70-4f31-8446-0669c0db71f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:57.561703 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848c09c3-3b70-4f31-8446-0669c0db71f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:57.563641 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848c09c3-3b70-4f31-8446-0669c0db71f9 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:57.580367 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3662cd0-46fc-47c8-8eee-44a549004f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:57.581159 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3662cd0-46fc-47c8-8eee-44a549004f41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:57.587605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3662cd0-46fc-47c8-8eee-44a549004f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:57.587605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3662cd0-46fc-47c8-8eee-44a549004f41 admin admin] PATH_INFO: `/v3/projects/70d967fb1d25472fbb05e4269f78d4d0` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:57.587605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3662cd0-46fc-47c8-8eee-44a549004f41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70d967fb1d25472fbb05e4269f78d4d0)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:57.587605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3662cd0-46fc-47c8-8eee-44a549004f41 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:57.610601 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 928/1853] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:57 2022] POST /identity/v3/projects => generated 379 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:57.612536 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff014fb-6e81-4821-90b1-3a271ffcb66e None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:57.612536 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff014fb-6e81-4821-90b1-3a271ffcb66e None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:57.612860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff014fb-6e81-4821-90b1-3a271ffcb66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:57.625884 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:57.670201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b3662cd0-46fc-47c8-8eee-44a549004f41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70d967fb1d25472fbb05e4269f78d4d0', 'request_id': 'req-b3662cd0-46fc-47c8-8eee-44a549004f41'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:57.676162 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b3662cd0-46fc-47c8-8eee-44a549004f41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70d967fb1d25472fbb05e4269f78d4d0', 'request_id': 'req-b3662cd0-46fc-47c8-8eee-44a549004f41'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:57.676971 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-dff014fb-6e81-4821-90b1-3a271ffcb66e None None] MFA Rules not processed for user `d4f68089787b4e7f8fbbb65ecfafcec0`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:57.678438 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-dff014fb-6e81-4821-90b1-3a271ffcb66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:57.686030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b3662cd0-46fc-47c8-8eee-44a549004f41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70d967fb1d25472fbb05e4269f78d4d0', 'request_id': 'req-b3662cd0-46fc-47c8-8eee-44a549004f41'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:57.701934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b3662cd0-46fc-47c8-8eee-44a549004f41 admin admin] The token cache is being invalidate because project 70d967fb1d25472fbb05e4269f78d4d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:08:57.702443 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b3662cd0-46fc-47c8-8eee-44a549004f41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:57.705219 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b3662cd0-46fc-47c8-8eee-44a549004f41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:08:57.708914 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 926/1854] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:08:57 2022] DELETE /identity/v3/projects/70d967fb1d25472fbb05e4269f78d4d0 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:57.711073 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3662cd0-46fc-47c8-8eee-44a549004f41 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:57.715707 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b3662cd0-46fc-47c8-8eee-44a549004f41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:57.801839 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 929/1855] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:57.803165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff014fb-6e81-4821-90b1-3a271ffcb66e None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:57.804597 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-dff014fb-6e81-4821-90b1-3a271ffcb66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:57.827607 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3662cd0-46fc-47c8-8eee-44a549004f41 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:57.842871 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed4e6e-cef0-41c2-a85b-0e99245036b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:57.843578 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed4e6e-cef0-41c2-a85b-0e99245036b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:57.843885 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed4e6e-cef0-41c2-a85b-0e99245036b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:57.844192 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed4e6e-cef0-41c2-a85b-0e99245036b0 admin admin] PATH_INFO: `/v3/projects/3d66b31a768742318b4de6e0b3d74c4a/users/4e8e9f312c104af39cb20091fa4024ad/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:57.846619 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed4e6e-cef0-41c2-a85b-0e99245036b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d66b31a768742318b4de6e0b3d74c4a, user_id=4e8e9f312c104af39cb20091fa4024ad, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:57.867070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed4e6e-cef0-41c2-a85b-0e99245036b0 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:57.894990 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 927/1856] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:57 2022] PUT /identity/v3/projects/3d66b31a768742318b4de6e0b3d74c4a/users/4e8e9f312c104af39cb20091fa4024ad/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:57.896215 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084bd3f-dc51-46ef-8402-5a4c4d8f2907 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:57.896523 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084bd3f-dc51-46ef-8402-5a4c4d8f2907 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:57.896789 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084bd3f-dc51-46ef-8402-5a4c4d8f2907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:57.948606 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-4084bd3f-dc51-46ef-8402-5a4c4d8f2907 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:57.951551 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff014fb-6e81-4821-90b1-3a271ffcb66e None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:57.953253 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4084bd3f-dc51-46ef-8402-5a4c4d8f2907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:57.973941 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:58.068256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe9aec-1fea-4a8c-a4e3-8864521c087e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:58.068938 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe9aec-1fea-4a8c-a4e3-8864521c087e admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:58.069072 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 928/1857] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:58.078410 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe9aec-1fea-4a8c-a4e3-8864521c087e admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:58.078410 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe9aec-1fea-4a8c-a4e3-8864521c087e admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:58.078930 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084bd3f-dc51-46ef-8402-5a4c4d8f2907 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:58.079018 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe9aec-1fea-4a8c-a4e3-8864521c087e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:58.079095 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4084bd3f-dc51-46ef-8402-5a4c4d8f2907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:58.079176 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe9aec-1fea-4a8c-a4e3-8864521c087e admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:58.088371 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084bd3f-dc51-46ef-8402-5a4c4d8f2907 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:58.109742 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6e013-33e8-4605-bce6-fb10cfa714fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:58.110538 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6e013-33e8-4605-bce6-fb10cfa714fc admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:58.110815 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6e013-33e8-4605-bce6-fb10cfa714fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:58.111090 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6e013-33e8-4605-bce6-fb10cfa714fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:58.112153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6e013-33e8-4605-bce6-fb10cfa714fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:58.116493 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6e013-33e8-4605-bce6-fb10cfa714fc admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:58.137043 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 930/1858] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:57 2022] POST /identity/v3/users => generated 290 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:58.137043 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5e4e0-1bbb-455d-853d-cd9bb6984f90 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:58.137043 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5e4e0-1bbb-455d-853d-cd9bb6984f90 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:58.137043 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5e4e0-1bbb-455d-853d-cd9bb6984f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:58.180309 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 929/1859] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:58 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:58.183011 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce9a9b-645b-4e45-809e-3dcff209a44d None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:58.183775 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce9a9b-645b-4e45-809e-3dcff209a44d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:58.184298 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce9a9b-645b-4e45-809e-3dcff209a44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:58.217653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-5cc5e4e0-1bbb-455d-853d-cd9bb6984f90 None None] MFA Rules not processed for user `4e8e9f312c104af39cb20091fa4024ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:58.219236 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5cc5e4e0-1bbb-455d-853d-cd9bb6984f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:58.229480 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-03ce9a9b-645b-4e45-809e-3dcff209a44d None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:58.233632 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-03ce9a9b-645b-4e45-809e-3dcff209a44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:58.272948 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:58.383795 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 930/1860] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:58.383795 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce9a9b-645b-4e45-809e-3dcff209a44d None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:58.403244 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce9a9b-645b-4e45-809e-3dcff209a44d None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:58.417502 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30bc8c8-ed2e-42f3-8717-e9b7ab88fa84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:58.419738 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 931/1861] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:58 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:58.421853 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc5e4e0-1bbb-455d-853d-cd9bb6984f90 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:58.426799 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bc8c8-ed2e-42f3-8717-e9b7ab88fa84 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:58.426799 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bc8c8-ed2e-42f3-8717-e9b7ab88fa84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:58.427084 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bc8c8-ed2e-42f3-8717-e9b7ab88fa84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:58.427835 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30bc8c8-ed2e-42f3-8717-e9b7ab88fa84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:58.430299 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30bc8c8-ed2e-42f3-8717-e9b7ab88fa84 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:58.447988 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc5e4e0-1bbb-455d-853d-cd9bb6984f90 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:58.458386 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15a5b7-4d11-4268-97e1-d9c6de3bfd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:58.459172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15a5b7-4d11-4268-97e1-d9c6de3bfd6a admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:58.459572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15a5b7-4d11-4268-97e1-d9c6de3bfd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:58.459860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15a5b7-4d11-4268-97e1-d9c6de3bfd6a admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:58.460822 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 931/1862] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:58.461909 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30bc8c8-ed2e-42f3-8717-e9b7ab88fa84 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:58.462955 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e15a5b7-4d11-4268-97e1-d9c6de3bfd6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:58.465686 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e15a5b7-4d11-4268-97e1-d9c6de3bfd6a admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:58.483403 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30bc8c8-ed2e-42f3-8717-e9b7ab88fa84 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:58.498909 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b94dd45-b8ec-49f0-837c-18bc98afc987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:58.499230 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b94dd45-b8ec-49f0-837c-18bc98afc987 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:58.499317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b94dd45-b8ec-49f0-837c-18bc98afc987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:58.499572 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b94dd45-b8ec-49f0-837c-18bc98afc987 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:58.500292 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b94dd45-b8ec-49f0-837c-18bc98afc987 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:58.500903 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b94dd45-b8ec-49f0-837c-18bc98afc987 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:58.502845 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b94dd45-b8ec-49f0-837c-18bc98afc987 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:58.507391 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 932/1863] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:58 2022] POST /identity/v3/users => generated 293 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:58.507391 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15a5b7-4d11-4268-97e1-d9c6de3bfd6a admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:58.508532 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1e15a5b7-4d11-4268-97e1-d9c6de3bfd6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:58.535189 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 932/1864] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:08:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:58.539213 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b94dd45-b8ec-49f0-837c-18bc98afc987 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:58.563264 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b94dd45-b8ec-49f0-837c-18bc98afc987 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:58.570351 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28acf506-de88-447d-82a4-aa77b6bce855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:58.571108 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28acf506-de88-447d-82a4-aa77b6bce855 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:58.571385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28acf506-de88-447d-82a4-aa77b6bce855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:58.572099 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28acf506-de88-447d-82a4-aa77b6bce855 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:58.572492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28acf506-de88-447d-82a4-aa77b6bce855 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:58.574303 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28acf506-de88-447d-82a4-aa77b6bce855 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:58.590006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 933/1865] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:58.590006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28acf506-de88-447d-82a4-aa77b6bce855 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:58.598142 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28acf506-de88-447d-82a4-aa77b6bce855 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:58.603245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd367b29-b980-42f8-9b8d-28a0d65a4f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:58.603897 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd367b29-b980-42f8-9b8d-28a0d65a4f56 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:58.604175 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd367b29-b980-42f8-9b8d-28a0d65a4f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:58.604479 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd367b29-b980-42f8-9b8d-28a0d65a4f56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:58.604906 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15a5b7-4d11-4268-97e1-d9c6de3bfd6a admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:58.605171 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd367b29-b980-42f8-9b8d-28a0d65a4f56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:58.606776 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd367b29-b980-42f8-9b8d-28a0d65a4f56 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:58.610308 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc187f-6cd1-41ff-a6f2-47325e45171d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:58.610986 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc187f-6cd1-41ff-a6f2-47325e45171d service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:58.611320 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc187f-6cd1-41ff-a6f2-47325e45171d service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:58.612306 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc187f-6cd1-41ff-a6f2-47325e45171d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:58.634219 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 934/1866] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:58.638483 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd367b29-b980-42f8-9b8d-28a0d65a4f56 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:58.639770 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bc187f-6cd1-41ff-a6f2-47325e45171d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:58.641847 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bc187f-6cd1-41ff-a6f2-47325e45171d service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:58.649660 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd367b29-b980-42f8-9b8d-28a0d65a4f56 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:58.655847 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 933/1867] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:58 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:58.656781 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc187f-6cd1-41ff-a6f2-47325e45171d service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:58.658662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b533038-717a-4768-a9ec-f414585c011e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:58.659686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b533038-717a-4768-a9ec-f414585c011e admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:58.660384 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b533038-717a-4768-a9ec-f414585c011e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:58.660872 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b533038-717a-4768-a9ec-f414585c011e admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:58.661912 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b533038-717a-4768-a9ec-f414585c011e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:58.667149 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b533038-717a-4768-a9ec-f414585c011e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:58.696747 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc187f-6cd1-41ff-a6f2-47325e45171d service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:58.707421 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebda6e3-a855-449c-8078-f18f5fc669f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:58.708443 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebda6e3-a855-449c-8078-f18f5fc669f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:58.708795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebda6e3-a855-449c-8078-f18f5fc669f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:58.709103 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebda6e3-a855-449c-8078-f18f5fc669f4 admin admin] PATH_INFO: `/v3/projects/25c402960b534b4f8c808a3e10637c3d/users/97c54b95585a4ffa93502ab65f6579b4/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:58.724441 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:58.726171 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 935/1868] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:58 2022] POST /identity/v3/projects => generated 365 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:58.730215 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b533038-717a-4768-a9ec-f414585c011e admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:58.732972 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebda6e3-a855-449c-8078-f18f5fc669f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25c402960b534b4f8c808a3e10637c3d, user_id=97c54b95585a4ffa93502ab65f6579b4, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:58.736367 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebda6e3-a855-449c-8078-f18f5fc669f4 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:58.756108 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 934/1869] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:58 2022] PUT /identity/v3/projects/25c402960b534b4f8c808a3e10637c3d/users/97c54b95585a4ffa93502ab65f6579b4/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:58.757324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebda6e3-a855-449c-8078-f18f5fc669f4 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:58.836544 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b533038-717a-4768-a9ec-f414585c011e admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:58.854283 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebda6e3-a855-449c-8078-f18f5fc669f4 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:58.860612 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eef128-1f96-4d5b-af61-02e6be82e1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:58.861524 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eef128-1f96-4d5b-af61-02e6be82e1be admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:58.861656 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eef128-1f96-4d5b-af61-02e6be82e1be admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:58.861903 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eef128-1f96-4d5b-af61-02e6be82e1be admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:58.862764 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eef128-1f96-4d5b-af61-02e6be82e1be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:58.872768 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eef128-1f96-4d5b-af61-02e6be82e1be admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:58.899051 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 935/1870] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:58 2022] POST /identity/v3/users => generated 283 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:58.900636 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8134a931-25c1-4a84-a4c7-c3ad91ab5b27 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:58.900935 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8134a931-25c1-4a84-a4c7-c3ad91ab5b27 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:58.901231 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8134a931-25c1-4a84-a4c7-c3ad91ab5b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:58.918219 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47ee2c-212d-491e-9edd-0cdbcdcc8591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:58.920699 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47ee2c-212d-491e-9edd-0cdbcdcc8591 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:58.921111 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47ee2c-212d-491e-9edd-0cdbcdcc8591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:58.921307 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47ee2c-212d-491e-9edd-0cdbcdcc8591 admin admin] PATH_INFO: `/v3/projects/f7b5af4d77bb470ca6e16e1762c7fd1c/users/cc05646dd2174266a1615b9812d41b94/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:58.923915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:58.936128 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-8134a931-25c1-4a84-a4c7-c3ad91ab5b27 None None] MFA Rules not processed for user `97c54b95585a4ffa93502ab65f6579b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:58.937707 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-8134a931-25c1-4a84-a4c7-c3ad91ab5b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:58.944945 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da47ee2c-212d-491e-9edd-0cdbcdcc8591 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7b5af4d77bb470ca6e16e1762c7fd1c, user_id=cc05646dd2174266a1615b9812d41b94, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:58.948566 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da47ee2c-212d-491e-9edd-0cdbcdcc8591 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:58.968678 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 936/1871] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:58 2022] PUT /identity/v3/projects/f7b5af4d77bb470ca6e16e1762c7fd1c/users/cc05646dd2174266a1615b9812d41b94/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:58.969601 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47ee2c-212d-491e-9edd-0cdbcdcc8591 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:59.014965 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 936/1872] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:59.016681 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8134a931-25c1-4a84-a4c7-c3ad91ab5b27 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:59.043263 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47ee2c-212d-491e-9edd-0cdbcdcc8591 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:59.050099 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dbe32b-fb61-4724-9441-46d1a3b124f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:59.050612 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbe32b-fb61-4724-9441-46d1a3b124f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:59.050893 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbe32b-fb61-4724-9441-46d1a3b124f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:59.051204 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbe32b-fb61-4724-9441-46d1a3b124f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:59.051939 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dbe32b-fb61-4724-9441-46d1a3b124f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:59.053635 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dbe32b-fb61-4724-9441-46d1a3b124f8 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:59.068214 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 937/1873] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:59.069713 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dbe32b-fb61-4724-9441-46d1a3b124f8 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:59.080370 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dbe32b-fb61-4724-9441-46d1a3b124f8 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:59.085456 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8134a931-25c1-4a84-a4c7-c3ad91ab5b27 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:59.086861 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c75a5-b1e9-4afa-99a3-92d7f8775e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:59.087482 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c75a5-b1e9-4afa-99a3-92d7f8775e25 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:59.087757 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c75a5-b1e9-4afa-99a3-92d7f8775e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:59.088025 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c75a5-b1e9-4afa-99a3-92d7f8775e25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:59.088749 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c75a5-b1e9-4afa-99a3-92d7f8775e25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:59.090496 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c75a5-b1e9-4afa-99a3-92d7f8775e25 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:59.100664 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 938/1874] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:59.101603 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c75a5-b1e9-4afa-99a3-92d7f8775e25 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:59.104625 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:59.117731 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c75a5-b1e9-4afa-99a3-92d7f8775e25 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:59.122601 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb3e91-338c-4a8b-85a5-799c65cef554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:59.123229 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb3e91-338c-4a8b-85a5-799c65cef554 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:59.123487 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb3e91-338c-4a8b-85a5-799c65cef554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:59.123761 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb3e91-338c-4a8b-85a5-799c65cef554 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:59.124504 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb3e91-338c-4a8b-85a5-799c65cef554 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:59.126622 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb3e91-338c-4a8b-85a5-799c65cef554 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:59.144474 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5139e36c-b22c-4dbf-bfb6-0e11c4a58d53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:59.145541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5139e36c-b22c-4dbf-bfb6-0e11c4a58d53 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:59.145986 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5139e36c-b22c-4dbf-bfb6-0e11c4a58d53 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:59.146340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5139e36c-b22c-4dbf-bfb6-0e11c4a58d53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:59.149164 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5139e36c-b22c-4dbf-bfb6-0e11c4a58d53 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:59.162153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 939/1875] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:59 2022] POST /identity/v3/projects => generated 379 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:59.163149 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb3e91-338c-4a8b-85a5-799c65cef554 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:59.231543 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb3e91-338c-4a8b-85a5-799c65cef554 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:59.236433 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede425b-84ac-4a4d-9114-6b1eeb1eab1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:59.237286 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede425b-84ac-4a4d-9114-6b1eeb1eab1c admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:59.237576 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede425b-84ac-4a4d-9114-6b1eeb1eab1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:59.237860 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede425b-84ac-4a4d-9114-6b1eeb1eab1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:59.238567 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede425b-84ac-4a4d-9114-6b1eeb1eab1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:59.240305 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede425b-84ac-4a4d-9114-6b1eeb1eab1c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:59.241717 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5139e36c-b22c-4dbf-bfb6-0e11c4a58d53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:59.243161 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5139e36c-b22c-4dbf-bfb6-0e11c4a58d53 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:59.251582 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 940/1876] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:59.253843 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 937/1877] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:59 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:59.255422 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede425b-84ac-4a4d-9114-6b1eeb1eab1c admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:59.257262 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5139e36c-b22c-4dbf-bfb6-0e11c4a58d53 service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:59.268877 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede425b-84ac-4a4d-9114-6b1eeb1eab1c admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:59.274819 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a80c30-2cc1-4da4-8859-0c8643d986bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:59.275030 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5139e36c-b22c-4dbf-bfb6-0e11c4a58d53 service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:59.278446 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a80c30-2cc1-4da4-8859-0c8643d986bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:59.279474 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a80c30-2cc1-4da4-8859-0c8643d986bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:59.279868 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a80c30-2cc1-4da4-8859-0c8643d986bf admin admin] PATH_INFO: `/v3/projects/f7b5af4d77bb470ca6e16e1762c7fd1c/users/cc05646dd2174266a1615b9812d41b94/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:59.281687 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218aa3be-6aaf-4039-861f-c781a09b95aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:59.282455 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218aa3be-6aaf-4039-861f-c781a09b95aa admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:59.282674 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a80c30-2cc1-4da4-8859-0c8643d986bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7b5af4d77bb470ca6e16e1762c7fd1c, user_id=cc05646dd2174266a1615b9812d41b94, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:59.282787 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218aa3be-6aaf-4039-861f-c781a09b95aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:59.282993 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218aa3be-6aaf-4039-861f-c781a09b95aa admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:59.283662 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218aa3be-6aaf-4039-861f-c781a09b95aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:59.285722 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218aa3be-6aaf-4039-861f-c781a09b95aa admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:59.286570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a80c30-2cc1-4da4-8859-0c8643d986bf admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:59.300421 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 941/1878] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:59 2022] PUT /identity/v3/projects/f7b5af4d77bb470ca6e16e1762c7fd1c/users/cc05646dd2174266a1615b9812d41b94/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:59.301170 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a80c30-2cc1-4da4-8859-0c8643d986bf admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:59.320869 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 938/1879] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:59 2022] POST /identity/v3/users => generated 296 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:59.320869 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328a6a1-78fc-442e-a33d-d895edd6b368 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:59.320869 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328a6a1-78fc-442e-a33d-d895edd6b368 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:59.320869 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328a6a1-78fc-442e-a33d-d895edd6b368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:59.354179 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-6328a6a1-78fc-442e-a33d-d895edd6b368 None None] MFA Rules not processed for user `cc05646dd2174266a1615b9812d41b94`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:59.355654 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-6328a6a1-78fc-442e-a33d-d895edd6b368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:59.357002 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:59.424507 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a80c30-2cc1-4da4-8859-0c8643d986bf admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:59.429443 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad9c3c-956b-4d46-b4c7-af5322a6dd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:59.430497 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad9c3c-956b-4d46-b4c7-af5322a6dd5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:59.430978 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad9c3c-956b-4d46-b4c7-af5322a6dd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:59.431418 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad9c3c-956b-4d46-b4c7-af5322a6dd5a admin admin] PATH_INFO: `/v3/projects/f1674f04a12949ca92c7b519811b2b4c/users/1bc63b9c6f7d424a91c72c1f44e5c8ef/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:59.443546 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 939/1880] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:59 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:59.444600 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328a6a1-78fc-442e-a33d-d895edd6b368 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:59.453143 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad9c3c-956b-4d46-b4c7-af5322a6dd5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1674f04a12949ca92c7b519811b2b4c, user_id=1bc63b9c6f7d424a91c72c1f44e5c8ef, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:59.456021 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad9c3c-956b-4d46-b4c7-af5322a6dd5a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:59.460867 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328a6a1-78fc-442e-a33d-d895edd6b368 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:59.466438 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed7968-f5f9-4116-9620-12df9cbc0698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:59.467152 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed7968-f5f9-4116-9620-12df9cbc0698 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:59.467437 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed7968-f5f9-4116-9620-12df9cbc0698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:59.467754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed7968-f5f9-4116-9620-12df9cbc0698 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:59.468564 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ed7968-f5f9-4116-9620-12df9cbc0698 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:59.470847 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ed7968-f5f9-4116-9620-12df9cbc0698 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:59.475099 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 942/1881] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:59 2022] PUT /identity/v3/projects/f1674f04a12949ca92c7b519811b2b4c/users/1bc63b9c6f7d424a91c72c1f44e5c8ef/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:59.475854 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad9c3c-956b-4d46-b4c7-af5322a6dd5a admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:59.477556 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-43ad9c3c-956b-4d46-b4c7-af5322a6dd5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:59.487964 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 940/1882] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:59.489689 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b759b9c-e907-401c-8db9-bb3631756c39 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:59.489972 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b759b9c-e907-401c-8db9-bb3631756c39 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:59.490232 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b759b9c-e907-401c-8db9-bb3631756c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:59.527013 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-0b759b9c-e907-401c-8db9-bb3631756c39 None None] MFA Rules not processed for user `1bc63b9c6f7d424a91c72c1f44e5c8ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:59.528796 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0b759b9c-e907-401c-8db9-bb3631756c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:59.551812 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad9c3c-956b-4d46-b4c7-af5322a6dd5a admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:59.557191 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803163ab-29d4-45b0-b222-311a13941e65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:59.558019 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803163ab-29d4-45b0-b222-311a13941e65 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:59.558283 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803163ab-29d4-45b0-b222-311a13941e65 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:59.559486 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803163ab-29d4-45b0-b222-311a13941e65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:59.584862 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:59.669659 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803163ab-29d4-45b0-b222-311a13941e65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:59.671234 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803163ab-29d4-45b0-b222-311a13941e65 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:59.674025 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 941/1883] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:08:59 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:59.675555 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b759b9c-e907-401c-8db9-bb3631756c39 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:59.694574 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 943/1884] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:59 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:59.695651 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803163ab-29d4-45b0-b222-311a13941e65 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:59.706510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803163ab-29d4-45b0-b222-311a13941e65 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:59.712105 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338fc3d-804d-49fd-9268-80a6084c7735 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:59.712847 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338fc3d-804d-49fd-9268-80a6084c7735 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:59.713135 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338fc3d-804d-49fd-9268-80a6084c7735 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:59.713410 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338fc3d-804d-49fd-9268-80a6084c7735 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:59.728515 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c338fc3d-804d-49fd-9268-80a6084c7735 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:59.730344 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c338fc3d-804d-49fd-9268-80a6084c7735 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:59.743675 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 944/1885] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:59 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:59.792511 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b759b9c-e907-401c-8db9-bb3631756c39 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:59.804495 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e743f9-0f4b-4b70-b4d2-25cd41dde42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:59.805744 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e743f9-0f4b-4b70-b4d2-25cd41dde42c admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:59.806275 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e743f9-0f4b-4b70-b4d2-25cd41dde42c admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:59.806704 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e743f9-0f4b-4b70-b4d2-25cd41dde42c admin admin] PATH_INFO: `/v3/projects/9e9b669e93674c7e9d76ca3cb1bb08d6/users/c554168adba4462b94303f3a4318a119/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:59.830675 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e743f9-0f4b-4b70-b4d2-25cd41dde42c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e9b669e93674c7e9d76ca3cb1bb08d6, user_id=c554168adba4462b94303f3a4318a119, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:59.834347 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e743f9-0f4b-4b70-b4d2-25cd41dde42c admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:59.854973 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 942/1886] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:59 2022] PUT /identity/v3/projects/9e9b669e93674c7e9d76ca3cb1bb08d6/users/c554168adba4462b94303f3a4318a119/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:59.859555 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338fc3d-804d-49fd-9268-80a6084c7735 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:59.883824 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338fc3d-804d-49fd-9268-80a6084c7735 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:59.944416 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:59.959633 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd389b-a26d-4f36-9711-cc7ff329670f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:59.960415 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd389b-a26d-4f36-9711-cc7ff329670f admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:59.960674 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd389b-a26d-4f36-9711-cc7ff329670f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:59.960977 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd389b-a26d-4f36-9711-cc7ff329670f admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:59.961697 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbd389b-a26d-4f36-9711-cc7ff329670f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:59.963342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbd389b-a26d-4f36-9711-cc7ff329670f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:59.977122 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 945/1887] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:08:59.982143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e743f9-0f4b-4b70-b4d2-25cd41dde42c admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:59.994220 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e743f9-0f4b-4b70-b4d2-25cd41dde42c admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:59.999386 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c68a84-00f6-4c2b-b709-65faf9fa5ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:00.000263 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c68a84-00f6-4c2b-b709-65faf9fa5ddf admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:00.000642 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c68a84-00f6-4c2b-b709-65faf9fa5ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:00.000930 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c68a84-00f6-4c2b-b709-65faf9fa5ddf admin admin] PATH_INFO: `/v3/projects/9e9b669e93674c7e9d76ca3cb1bb08d6/users/c554168adba4462b94303f3a4318a119/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:00.002970 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c68a84-00f6-4c2b-b709-65faf9fa5ddf admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e9b669e93674c7e9d76ca3cb1bb08d6, user_id=c554168adba4462b94303f3a4318a119, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:00.005705 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c68a84-00f6-4c2b-b709-65faf9fa5ddf admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:00.028309 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 943/1888] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:59 2022] PUT /identity/v3/projects/9e9b669e93674c7e9d76ca3cb1bb08d6/users/c554168adba4462b94303f3a4318a119/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:00.034180 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510348eb-dbb1-4385-bf0f-9adc493850eb None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:00.034662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510348eb-dbb1-4385-bf0f-9adc493850eb None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:00.035022 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510348eb-dbb1-4385-bf0f-9adc493850eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:00.072045 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-510348eb-dbb1-4385-bf0f-9adc493850eb None None] MFA Rules not processed for user `c554168adba4462b94303f3a4318a119`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:00.073541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-510348eb-dbb1-4385-bf0f-9adc493850eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:00.172303 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 946/1889] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:00 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:00.197812 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c68a84-00f6-4c2b-b709-65faf9fa5ddf admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:00.281194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c68a84-00f6-4c2b-b709-65faf9fa5ddf admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:00.288109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7413284a-86ca-4bd0-a70c-bd7ef79098c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:00.288910 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413284a-86ca-4bd0-a70c-bd7ef79098c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:00.289204 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510348eb-dbb1-4385-bf0f-9adc493850eb None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:00.289674 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413284a-86ca-4bd0-a70c-bd7ef79098c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:00.289872 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413284a-86ca-4bd0-a70c-bd7ef79098c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:00.302647 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7413284a-86ca-4bd0-a70c-bd7ef79098c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:00.304123 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7413284a-86ca-4bd0-a70c-bd7ef79098c1 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:00.315534 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 944/1890] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:00 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:00.358854 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510348eb-dbb1-4385-bf0f-9adc493850eb None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:00.359645 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7413284a-86ca-4bd0-a70c-bd7ef79098c1 service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:00.364004 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386674dc-3ab7-49dc-8c79-7f5a35eb5e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:00.364727 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386674dc-3ab7-49dc-8c79-7f5a35eb5e16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:00.365002 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386674dc-3ab7-49dc-8c79-7f5a35eb5e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:00.365276 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386674dc-3ab7-49dc-8c79-7f5a35eb5e16 admin admin] PATH_INFO: `/v3/users/d4f68089787b4e7f8fbbb65ecfafcec0` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:00.366497 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386674dc-3ab7-49dc-8c79-7f5a35eb5e16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4f68089787b4e7f8fbbb65ecfafcec0)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:00.368476 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386674dc-3ab7-49dc-8c79-7f5a35eb5e16 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:00.369003 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7413284a-86ca-4bd0-a70c-bd7ef79098c1 service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:00.374863 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51bd71-b60c-422c-b7f4-e5d7e793fc44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:00.375522 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51bd71-b60c-422c-b7f4-e5d7e793fc44 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:00.375801 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51bd71-b60c-422c-b7f4-e5d7e793fc44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:00.376051 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51bd71-b60c-422c-b7f4-e5d7e793fc44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:00.378177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ab51bd71-b60c-422c-b7f4-e5d7e793fc44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:00.427935 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:00.432691 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-386674dc-3ab7-49dc-8c79-7f5a35eb5e16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4f68089787b4e7f8fbbb65ecfafcec0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:00.433956 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-386674dc-3ab7-49dc-8c79-7f5a35eb5e16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4f68089787b4e7f8fbbb65ecfafcec0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:00.454552 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-386674dc-3ab7-49dc-8c79-7f5a35eb5e16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4f68089787b4e7f8fbbb65ecfafcec0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:00.460542 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:00.462771 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-386674dc-3ab7-49dc-8c79-7f5a35eb5e16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4f68089787b4e7f8fbbb65ecfafcec0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:00.463583 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-386674dc-3ab7-49dc-8c79-7f5a35eb5e16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4f68089787b4e7f8fbbb65ecfafcec0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:00.468027 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51bd71-b60c-422c-b7f4-e5d7e793fc44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:00.469605 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51bd71-b60c-422c-b7f4-e5d7e793fc44 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:00.481004 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 945/1891] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:00 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:09:00.485028 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 947/1892] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:00 2022] DELETE /identity/v3/users/d4f68089787b4e7f8fbbb65ecfafcec0 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:00.493726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44213a-d57b-40d3-b113-dfa7e635c726 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:00.494008 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44213a-d57b-40d3-b113-dfa7e635c726 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:00.494276 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44213a-d57b-40d3-b113-dfa7e635c726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:00.531854 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-2c44213a-d57b-40d3-b113-dfa7e635c726 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:00.534097 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-2c44213a-d57b-40d3-b113-dfa7e635c726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:00.625590 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 946/1893] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:00.654362 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386674dc-3ab7-49dc-8c79-7f5a35eb5e16 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:00.656312 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-386674dc-3ab7-49dc-8c79-7f5a35eb5e16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:00.774247 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386674dc-3ab7-49dc-8c79-7f5a35eb5e16 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:00.798590 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc2a44-d6b1-41f0-85c7-5a68c8361774 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:00.800251 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc2a44-d6b1-41f0-85c7-5a68c8361774 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:00.800574 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc2a44-d6b1-41f0-85c7-5a68c8361774 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:00.800855 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc2a44-d6b1-41f0-85c7-5a68c8361774 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:00.829672 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bc2a44-d6b1-41f0-85c7-5a68c8361774 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:00.831341 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bc2a44-d6b1-41f0-85c7-5a68c8361774 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:00.851867 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 948/1894] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:09:01.491107 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c44213a-d57b-40d3-b113-dfa7e635c726 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:01.492617 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-2c44213a-d57b-40d3-b113-dfa7e635c726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:01.508447 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c44213a-d57b-40d3-b113-dfa7e635c726 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:01.513495 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0d689-4878-4571-9f3e-f76d37a00927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:01.514244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0d689-4878-4571-9f3e-f76d37a00927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:01.514609 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0d689-4878-4571-9f3e-f76d37a00927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:01.514879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0d689-4878-4571-9f3e-f76d37a00927 admin admin] PATH_INFO: `/v3/projects/ac97d680a31b42aa978d067b7af07d24` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:01.516324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e0d689-4878-4571-9f3e-f76d37a00927 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac97d680a31b42aa978d067b7af07d24)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:01.519189 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e0d689-4878-4571-9f3e-f76d37a00927 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:01.605355 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a5e0d689-4878-4571-9f3e-f76d37a00927 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac97d680a31b42aa978d067b7af07d24', 'request_id': 'req-a5e0d689-4878-4571-9f3e-f76d37a00927'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:01.614591 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a5e0d689-4878-4571-9f3e-f76d37a00927 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac97d680a31b42aa978d067b7af07d24', 'request_id': 'req-a5e0d689-4878-4571-9f3e-f76d37a00927'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:01.625788 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a5e0d689-4878-4571-9f3e-f76d37a00927 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac97d680a31b42aa978d067b7af07d24', 'request_id': 'req-a5e0d689-4878-4571-9f3e-f76d37a00927'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:01.648070 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a5e0d689-4878-4571-9f3e-f76d37a00927 admin admin] The token cache is being invalidate because project ac97d680a31b42aa978d067b7af07d24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:01.648481 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a5e0d689-4878-4571-9f3e-f76d37a00927 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:01.649455 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a5e0d689-4878-4571-9f3e-f76d37a00927 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:01.655787 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 947/1895] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:01 2022] DELETE /identity/v3/projects/ac97d680a31b42aa978d067b7af07d24 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:01.657407 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc2a44-d6b1-41f0-85c7-5a68c8361774 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:01.673030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc2a44-d6b1-41f0-85c7-5a68c8361774 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:01.743766 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10abbb-d0e3-450e-a15b-257df0d9573c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:01.744475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10abbb-d0e3-450e-a15b-257df0d9573c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:01.744898 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10abbb-d0e3-450e-a15b-257df0d9573c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:01.745212 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10abbb-d0e3-450e-a15b-257df0d9573c admin admin] PATH_INFO: `/v3/users/cc05646dd2174266a1615b9812d41b94` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:01.746667 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a10abbb-d0e3-450e-a15b-257df0d9573c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc05646dd2174266a1615b9812d41b94)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:01.749059 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a10abbb-d0e3-450e-a15b-257df0d9573c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:01.847206 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-9a10abbb-d0e3-450e-a15b-257df0d9573c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc05646dd2174266a1615b9812d41b94'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:01.848027 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-9a10abbb-d0e3-450e-a15b-257df0d9573c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc05646dd2174266a1615b9812d41b94'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:01.870893 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-9a10abbb-d0e3-450e-a15b-257df0d9573c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc05646dd2174266a1615b9812d41b94'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:01.888660 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-9a10abbb-d0e3-450e-a15b-257df0d9573c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc05646dd2174266a1615b9812d41b94'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:01.891059 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-9a10abbb-d0e3-450e-a15b-257df0d9573c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc05646dd2174266a1615b9812d41b94'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:01.935217 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 949/1896] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:01 2022] DELETE /identity/v3/users/cc05646dd2174266a1615b9812d41b94 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:02.561022 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0d689-4878-4571-9f3e-f76d37a00927 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:02.562846 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a5e0d689-4878-4571-9f3e-f76d37a00927 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:02.582764 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0d689-4878-4571-9f3e-f76d37a00927 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:02.645700 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:02.652516 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e34482-79b4-4d27-a597-29fbe049a75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:02.653749 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e34482-79b4-4d27-a597-29fbe049a75e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:02.654348 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e34482-79b4-4d27-a597-29fbe049a75e admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:02.654820 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e34482-79b4-4d27-a597-29fbe049a75e admin admin] PATH_INFO: `/v3/projects/f7b5af4d77bb470ca6e16e1762c7fd1c` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:02.656824 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e34482-79b4-4d27-a597-29fbe049a75e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7b5af4d77bb470ca6e16e1762c7fd1c)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:02.659234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e34482-79b4-4d27-a597-29fbe049a75e admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:02.710407 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-f6e34482-79b4-4d27-a597-29fbe049a75e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7b5af4d77bb470ca6e16e1762c7fd1c', 'request_id': 'req-f6e34482-79b4-4d27-a597-29fbe049a75e'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:02.717244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-f6e34482-79b4-4d27-a597-29fbe049a75e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7b5af4d77bb470ca6e16e1762c7fd1c', 'request_id': 'req-f6e34482-79b4-4d27-a597-29fbe049a75e'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:02.726258 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-f6e34482-79b4-4d27-a597-29fbe049a75e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7b5af4d77bb470ca6e16e1762c7fd1c', 'request_id': 'req-f6e34482-79b4-4d27-a597-29fbe049a75e'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:02.741311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-f6e34482-79b4-4d27-a597-29fbe049a75e admin admin] The token cache is being invalidate because project f7b5af4d77bb470ca6e16e1762c7fd1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:02.741813 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-f6e34482-79b4-4d27-a597-29fbe049a75e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:02.742881 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-f6e34482-79b4-4d27-a597-29fbe049a75e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:02.745678 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 948/1897] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:02 2022] DELETE /identity/v3/projects/f7b5af4d77bb470ca6e16e1762c7fd1c => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:02.755843 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd77db-eca0-4056-b576-f192c314bb73 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:02.755843 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd77db-eca0-4056-b576-f192c314bb73 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:02.756455 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd77db-eca0-4056-b576-f192c314bb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:02.786023 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-88fd77db-eca0-4056-b576-f192c314bb73 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:02.787253 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-88fd77db-eca0-4056-b576-f192c314bb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:02.793867 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:02.858551 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 950/1898] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:02.861220 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c1371-7a54-49ff-adc1-b492b792d2c1 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:02.861514 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c1371-7a54-49ff-adc1-b492b792d2c1 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:02.861864 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c1371-7a54-49ff-adc1-b492b792d2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:02.893201 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-540c1371-7a54-49ff-adc1-b492b792d2c1 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:02.894651 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-540c1371-7a54-49ff-adc1-b492b792d2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:02.904300 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 949/1899] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:02.909176 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fd77db-eca0-4056-b576-f192c314bb73 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:02.919874 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fd77db-eca0-4056-b576-f192c314bb73 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:02.926431 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0995ad7-a1af-45f1-8ec9-1c852801e81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:02.927090 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0995ad7-a1af-45f1-8ec9-1c852801e81b admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:02.927370 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0995ad7-a1af-45f1-8ec9-1c852801e81b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:02.930067 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0995ad7-a1af-45f1-8ec9-1c852801e81b admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:02.930884 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0995ad7-a1af-45f1-8ec9-1c852801e81b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:02.931189 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0995ad7-a1af-45f1-8ec9-1c852801e81b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:02.932913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0995ad7-a1af-45f1-8ec9-1c852801e81b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:02.946915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 951/1900] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:02.951856 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c1371-7a54-49ff-adc1-b492b792d2c1 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:02.962526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c1371-7a54-49ff-adc1-b492b792d2c1 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:02.968548 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84526d7d-9ce7-4991-a8a0-5b8002a2404c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:02.969267 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84526d7d-9ce7-4991-a8a0-5b8002a2404c admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:02.969579 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84526d7d-9ce7-4991-a8a0-5b8002a2404c admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:02.969868 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84526d7d-9ce7-4991-a8a0-5b8002a2404c admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:02.970593 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84526d7d-9ce7-4991-a8a0-5b8002a2404c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:02.972767 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84526d7d-9ce7-4991-a8a0-5b8002a2404c admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:03.010131 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 950/1901] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:02 2022] POST /identity/v3/projects => generated 369 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:03.011321 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0995ad7-a1af-45f1-8ec9-1c852801e81b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:03.023847 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0995ad7-a1af-45f1-8ec9-1c852801e81b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:03.105444 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90165f15-5796-4a01-a834-f5012565e952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:03.106545 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90165f15-5796-4a01-a834-f5012565e952 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:03.107045 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90165f15-5796-4a01-a834-f5012565e952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:03.107539 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90165f15-5796-4a01-a834-f5012565e952 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:03.108861 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90165f15-5796-4a01-a834-f5012565e952 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:03.112887 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90165f15-5796-4a01-a834-f5012565e952 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:03.145759 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 952/1902] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:03 2022] POST /identity/v3/users => generated 285 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:03.150245 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84526d7d-9ce7-4991-a8a0-5b8002a2404c admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:03.162376 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84526d7d-9ce7-4991-a8a0-5b8002a2404c admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:03.168054 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5dd439-7545-40fa-be15-a13fdc4f0c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:03.169092 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5dd439-7545-40fa-be15-a13fdc4f0c6a admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:03.169365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5dd439-7545-40fa-be15-a13fdc4f0c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:03.169578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5dd439-7545-40fa-be15-a13fdc4f0c6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:03.170458 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5dd439-7545-40fa-be15-a13fdc4f0c6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:03.172721 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5dd439-7545-40fa-be15-a13fdc4f0c6a admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:03.185999 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 951/1903] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:03.190490 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90165f15-5796-4a01-a834-f5012565e952 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:03.200979 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90165f15-5796-4a01-a834-f5012565e952 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:03.206046 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325d16f-109c-47fe-8011-2fd63e96fb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:03.206714 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325d16f-109c-47fe-8011-2fd63e96fb81 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:03.206988 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325d16f-109c-47fe-8011-2fd63e96fb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:03.207258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325d16f-109c-47fe-8011-2fd63e96fb81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:03.207950 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325d16f-109c-47fe-8011-2fd63e96fb81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:03.209589 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325d16f-109c-47fe-8011-2fd63e96fb81 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:03.228412 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 953/1904] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:03.229835 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5dd439-7545-40fa-be15-a13fdc4f0c6a admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:03.240153 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5dd439-7545-40fa-be15-a13fdc4f0c6a admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:03.244800 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8edd377-14e6-49f7-a52b-327a9046f31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:03.245471 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edd377-14e6-49f7-a52b-327a9046f31d admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:03.245747 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edd377-14e6-49f7-a52b-327a9046f31d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:03.246022 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edd377-14e6-49f7-a52b-327a9046f31d admin admin] PATH_INFO: `/v3/projects/3f8c3fa7783a444a9883efe4ec0c6d8d/users/ae040996063243a5866ba35432b4af44/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:03.261370 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8edd377-14e6-49f7-a52b-327a9046f31d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f8c3fa7783a444a9883efe4ec0c6d8d, user_id=ae040996063243a5866ba35432b4af44, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:03.264029 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8edd377-14e6-49f7-a52b-327a9046f31d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:03.281941 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 952/1905] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:03 2022] PUT /identity/v3/projects/3f8c3fa7783a444a9883efe4ec0c6d8d/users/ae040996063243a5866ba35432b4af44/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:03.300542 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ba4be-3a4e-46e7-b0aa-4ee389144ab1 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:03.300542 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ba4be-3a4e-46e7-b0aa-4ee389144ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:03.300542 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ba4be-3a4e-46e7-b0aa-4ee389144ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:03.330488 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-814ba4be-3a4e-46e7-b0aa-4ee389144ab1 None None] MFA Rules not processed for user `ae040996063243a5866ba35432b4af44`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:03.331891 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-814ba4be-3a4e-46e7-b0aa-4ee389144ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:03.412321 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 954/1906] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:03 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:03.416881 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8edd377-14e6-49f7-a52b-327a9046f31d admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:03.483747 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8edd377-14e6-49f7-a52b-327a9046f31d admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:03.488803 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fba5e3-9210-4d09-ab9b-85a960358452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:03.489442 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fba5e3-9210-4d09-ab9b-85a960358452 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:03.489725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fba5e3-9210-4d09-ab9b-85a960358452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:03.490547 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fba5e3-9210-4d09-ab9b-85a960358452 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:03.492017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fba5e3-9210-4d09-ab9b-85a960358452 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:03.495718 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fba5e3-9210-4d09-ab9b-85a960358452 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:03.526265 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 953/1907] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:03 2022] POST /identity/v3/projects => generated 369 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:03.531322 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ba4be-3a4e-46e7-b0aa-4ee389144ab1 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:03.543232 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ba4be-3a4e-46e7-b0aa-4ee389144ab1 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:03.552204 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:03.603381 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43a1e8-ded2-478f-8951-ffe2a2edaef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:03.604036 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43a1e8-ded2-478f-8951-ffe2a2edaef4 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:03.604324 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43a1e8-ded2-478f-8951-ffe2a2edaef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:03.604614 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43a1e8-ded2-478f-8951-ffe2a2edaef4 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:03.605278 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43a1e8-ded2-478f-8951-ffe2a2edaef4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:03.607206 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43a1e8-ded2-478f-8951-ffe2a2edaef4 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:03.630784 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 955/1908] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:03 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:03.635063 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fba5e3-9210-4d09-ab9b-85a960358452 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:03.645614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fba5e3-9210-4d09-ab9b-85a960358452 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:03.650542 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a67ae6-6be8-4522-bb05-29d501b10912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:03.651150 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a67ae6-6be8-4522-bb05-29d501b10912 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:03.651602 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a67ae6-6be8-4522-bb05-29d501b10912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:03.651875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a67ae6-6be8-4522-bb05-29d501b10912 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:03.652658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a67ae6-6be8-4522-bb05-29d501b10912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:03.654634 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a67ae6-6be8-4522-bb05-29d501b10912 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:03.657116 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:03.675556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 954/1909] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:03.676784 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43a1e8-ded2-478f-8951-ffe2a2edaef4 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:03.687475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43a1e8-ded2-478f-8951-ffe2a2edaef4 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:03.692838 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630a3d1-c8f4-48f7-9b15-97d212123f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:03.693538 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630a3d1-c8f4-48f7-9b15-97d212123f43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:03.693835 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630a3d1-c8f4-48f7-9b15-97d212123f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:03.694110 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630a3d1-c8f4-48f7-9b15-97d212123f43 admin admin] PATH_INFO: `/v3/projects/1ae1a7528bfd4f348ddbad51a65a2604/users/5c9aeef001374bd5a22a8a0fc8ca3f61/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:03.718448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8630a3d1-c8f4-48f7-9b15-97d212123f43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ae1a7528bfd4f348ddbad51a65a2604, user_id=5c9aeef001374bd5a22a8a0fc8ca3f61, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:03.721290 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8630a3d1-c8f4-48f7-9b15-97d212123f43 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:03.737977 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 956/1910] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:03 2022] PUT /identity/v3/projects/1ae1a7528bfd4f348ddbad51a65a2604/users/5c9aeef001374bd5a22a8a0fc8ca3f61/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:03.742078 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a67ae6-6be8-4522-bb05-29d501b10912 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:03.752515 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a67ae6-6be8-4522-bb05-29d501b10912 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:03.813469 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f6704-8fa2-4978-88b9-339990e3acfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:03.814100 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f6704-8fa2-4978-88b9-339990e3acfd admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:03.814383 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f6704-8fa2-4978-88b9-339990e3acfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:03.814658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f6704-8fa2-4978-88b9-339990e3acfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:03.815413 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218f6704-8fa2-4978-88b9-339990e3acfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:03.817072 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218f6704-8fa2-4978-88b9-339990e3acfd admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:03.827268 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 955/1911] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:03 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:03.833537 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630a3d1-c8f4-48f7-9b15-97d212123f43 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:03.850384 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630a3d1-c8f4-48f7-9b15-97d212123f43 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:03.855624 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8806c02-e11d-4987-9457-e01627f4749a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:03.856479 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8806c02-e11d-4987-9457-e01627f4749a admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:03.856758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8806c02-e11d-4987-9457-e01627f4749a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:03.857027 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8806c02-e11d-4987-9457-e01627f4749a admin admin] PATH_INFO: `/v3/projects/1ae1a7528bfd4f348ddbad51a65a2604/users/5c9aeef001374bd5a22a8a0fc8ca3f61/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:03.859612 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8806c02-e11d-4987-9457-e01627f4749a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ae1a7528bfd4f348ddbad51a65a2604, user_id=5c9aeef001374bd5a22a8a0fc8ca3f61, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:03.862944 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8806c02-e11d-4987-9457-e01627f4749a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:03.874942 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 957/1912] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:03 2022] PUT /identity/v3/projects/1ae1a7528bfd4f348ddbad51a65a2604/users/5c9aeef001374bd5a22a8a0fc8ca3f61/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:03.886333 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27019b58-c4e4-4316-adff-2f8bb70cb0dd None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:03.886664 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27019b58-c4e4-4316-adff-2f8bb70cb0dd None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:03.886819 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27019b58-c4e4-4316-adff-2f8bb70cb0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:03.918933 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-27019b58-c4e4-4316-adff-2f8bb70cb0dd None None] MFA Rules not processed for user `5c9aeef001374bd5a22a8a0fc8ca3f61`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:03.920183 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-27019b58-c4e4-4316-adff-2f8bb70cb0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:03.992556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 956/1913] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:03 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:04.007592 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8806c02-e11d-4987-9457-e01627f4749a admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:04.009546 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b8806c02-e11d-4987-9457-e01627f4749a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:04.077578 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8806c02-e11d-4987-9457-e01627f4749a admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:04.081705 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6737d0-4a53-4afc-9d74-008c630b6c71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:04.082364 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6737d0-4a53-4afc-9d74-008c630b6c71 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:04.082640 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6737d0-4a53-4afc-9d74-008c630b6c71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:04.082912 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6737d0-4a53-4afc-9d74-008c630b6c71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:04.092899 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6737d0-4a53-4afc-9d74-008c630b6c71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:04.094528 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6737d0-4a53-4afc-9d74-008c630b6c71 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:04.104316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 958/1914] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:04 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:05.724367 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27019b58-c4e4-4316-adff-2f8bb70cb0dd None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:05.808587 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27019b58-c4e4-4316-adff-2f8bb70cb0dd None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:05.818335 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8fe2e4-1ab6-43c5-ac7d-0b1d39299a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:05.819087 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fe2e4-1ab6-43c5-ac7d-0b1d39299a04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:05.819854 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fe2e4-1ab6-43c5-ac7d-0b1d39299a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:05.820134 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fe2e4-1ab6-43c5-ac7d-0b1d39299a04 admin admin] PATH_INFO: `/v3/users/ae040996063243a5866ba35432b4af44` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:05.821512 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8fe2e4-1ab6-43c5-ac7d-0b1d39299a04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae040996063243a5866ba35432b4af44)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:05.824126 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8fe2e4-1ab6-43c5-ac7d-0b1d39299a04 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:05.866413 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:05.928725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1e8fe2e4-1ab6-43c5-ac7d-0b1d39299a04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae040996063243a5866ba35432b4af44'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:05.930806 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1e8fe2e4-1ab6-43c5-ac7d-0b1d39299a04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae040996063243a5866ba35432b4af44'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:05.945639 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1e8fe2e4-1ab6-43c5-ac7d-0b1d39299a04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae040996063243a5866ba35432b4af44'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:05.958963 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1e8fe2e4-1ab6-43c5-ac7d-0b1d39299a04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae040996063243a5866ba35432b4af44'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:05.970021 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6737d0-4a53-4afc-9d74-008c630b6c71 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:05.972500 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1c6737d0-4a53-4afc-9d74-008c630b6c71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:05.986755 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1e8fe2e4-1ab6-43c5-ac7d-0b1d39299a04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae040996063243a5866ba35432b4af44'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:05.993808 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6737d0-4a53-4afc-9d74-008c630b6c71 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:06.004915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c14c21-8191-4eb5-8737-9dd8579ff767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:06.006091 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c14c21-8191-4eb5-8737-9dd8579ff767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:06.006491 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c14c21-8191-4eb5-8737-9dd8579ff767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:06.006909 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c14c21-8191-4eb5-8737-9dd8579ff767 admin admin] PATH_INFO: `/v3/users/1423bab4b1cd4ac5acc7ebe55b1f27ef` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:06.008515 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c14c21-8191-4eb5-8737-9dd8579ff767 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1423bab4b1cd4ac5acc7ebe55b1f27ef)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:06.011510 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c14c21-8191-4eb5-8737-9dd8579ff767 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:06.016627 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 957/1915] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:05 2022] DELETE /identity/v3/users/ae040996063243a5866ba35432b4af44 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:06.024874 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e45c43-a02d-40c6-8a75-f143c8a5842e None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:06.025553 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e45c43-a02d-40c6-8a75-f143c8a5842e None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:06.027373 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e45c43-a02d-40c6-8a75-f143c8a5842e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:06.081449 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-49e45c43-a02d-40c6-8a75-f143c8a5842e None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:06.088037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-49e45c43-a02d-40c6-8a75-f143c8a5842e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:06.126835 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-77c14c21-8191-4eb5-8737-9dd8579ff767 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1423bab4b1cd4ac5acc7ebe55b1f27ef'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:06.127539 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-77c14c21-8191-4eb5-8737-9dd8579ff767 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1423bab4b1cd4ac5acc7ebe55b1f27ef'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:06.137662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:06.175026 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-77c14c21-8191-4eb5-8737-9dd8579ff767 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1423bab4b1cd4ac5acc7ebe55b1f27ef'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:06.200921 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-77c14c21-8191-4eb5-8737-9dd8579ff767 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1423bab4b1cd4ac5acc7ebe55b1f27ef'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:06.201971 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-77c14c21-8191-4eb5-8737-9dd8579ff767 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1423bab4b1cd4ac5acc7ebe55b1f27ef'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:06.224643 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 958/1916] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:06.226778 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 959/1917] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:05 2022] DELETE /identity/v3/users/1423bab4b1cd4ac5acc7ebe55b1f27ef => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:06.232218 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67845a-4b46-4840-baec-98dd06200715 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:06.232685 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67845a-4b46-4840-baec-98dd06200715 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:06.233217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67845a-4b46-4840-baec-98dd06200715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:06.261990 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-6d67845a-4b46-4840-baec-98dd06200715 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:06.264700 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-6d67845a-4b46-4840-baec-98dd06200715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:06.299938 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c14c21-8191-4eb5-8737-9dd8579ff767 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:06.301870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-77c14c21-8191-4eb5-8737-9dd8579ff767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:06.376446 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 959/1918] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:06.392140 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c14c21-8191-4eb5-8737-9dd8579ff767 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:06.397356 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c706bd-e2b6-4505-9b6b-ea34c6f48300 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:06.398317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c706bd-e2b6-4505-9b6b-ea34c6f48300 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:06.398843 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c706bd-e2b6-4505-9b6b-ea34c6f48300 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:06.399424 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c706bd-e2b6-4505-9b6b-ea34c6f48300 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:06.411535 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d67845a-4b46-4840-baec-98dd06200715 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:06.425014 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d67845a-4b46-4840-baec-98dd06200715 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:06.430351 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba439d7-bc49-4001-96c7-bf006c460087 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:06.430864 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba439d7-bc49-4001-96c7-bf006c460087 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:06.431147 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba439d7-bc49-4001-96c7-bf006c460087 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:06.431460 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c706bd-e2b6-4505-9b6b-ea34c6f48300 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:06.431638 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba439d7-bc49-4001-96c7-bf006c460087 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:06.436205 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c706bd-e2b6-4505-9b6b-ea34c6f48300 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:06.452610 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba439d7-bc49-4001-96c7-bf006c460087 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:06.454675 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba439d7-bc49-4001-96c7-bf006c460087 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:06.469230 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 960/1919] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:06.481753 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 960/1920] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:07.321719 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095468f-ce6f-443d-ad8e-0bba84a18ca6 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:07.322337 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095468f-ce6f-443d-ad8e-0bba84a18ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:07.322625 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095468f-ce6f-443d-ad8e-0bba84a18ca6 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:07.324389 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 961/1921] 10.222.0.45 () {54 vars in 921 bytes} [Mon Aug 1 14:09:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:09:07.340436 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce3ec2-bc45-4a70-83e7-e88f292929f8 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:07.340910 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce3ec2-bc45-4a70-83e7-e88f292929f8 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:07.341493 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce3ec2-bc45-4a70-83e7-e88f292929f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:07.378137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-9bce3ec2-bc45-4a70-83e7-e88f292929f8 None None] MFA Rules not processed for user `5b3adc14dcaf4ac79b43bb97fa1ba1d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:07.380874 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-9bce3ec2-bc45-4a70-83e7-e88f292929f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:07.456916 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:07.542738 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 961/1922] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Aug 1 14:09:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:08.201166 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095468f-ce6f-443d-ad8e-0bba84a18ca6 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:08.204121 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1095468f-ce6f-443d-ad8e-0bba84a18ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:08.234489 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095468f-ce6f-443d-ad8e-0bba84a18ca6 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:08.255139 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ec6bb-a971-44fd-b2b3-8627e14cfdb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:08.257148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ec6bb-a971-44fd-b2b3-8627e14cfdb7 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:08.258044 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ec6bb-a971-44fd-b2b3-8627e14cfdb7 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:08.260121 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ec6bb-a971-44fd-b2b3-8627e14cfdb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:08.280127 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce3ec2-bc45-4a70-83e7-e88f292929f8 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:08.282853 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-9bce3ec2-bc45-4a70-83e7-e88f292929f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:08.292785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ec6bb-a971-44fd-b2b3-8627e14cfdb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:08.294422 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ec6bb-a971-44fd-b2b3-8627e14cfdb7 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:08.315861 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 962/1923] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:08.316654 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ec6bb-a971-44fd-b2b3-8627e14cfdb7 service nova] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:08.323290 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ef1ec6bb-a971-44fd-b2b3-8627e14cfdb7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:08.328723 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce3ec2-bc45-4a70-83e7-e88f292929f8 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:08.347779 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7e85d-f591-40ae-963e-295c51c78108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:08.350303 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7e85d-f591-40ae-963e-295c51c78108 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:08.357007 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7e85d-f591-40ae-963e-295c51c78108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:08.357504 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7e85d-f591-40ae-963e-295c51c78108 admin admin] PATH_INFO: `/v3/projects/3f8c3fa7783a444a9883efe4ec0c6d8d` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:08.359616 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e7e85d-f591-40ae-963e-295c51c78108 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f8c3fa7783a444a9883efe4ec0c6d8d)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:08.361495 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ec6bb-a971-44fd-b2b3-8627e14cfdb7 service nova] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:08.365081 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e7e85d-f591-40ae-963e-295c51c78108 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:08.385443 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa3ef7c-2670-4b4c-ab5b-a2f2fdf4453c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:08.386114 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3ef7c-2670-4b4c-ab5b-a2f2fdf4453c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:08.386389 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3ef7c-2670-4b4c-ab5b-a2f2fdf4453c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:08.386669 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3ef7c-2670-4b4c-ab5b-a2f2fdf4453c admin admin] PATH_INFO: `/v3/projects/8a6a587aedea4024aa5cf14dbe3f51e5` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:08.393182 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa3ef7c-2670-4b4c-ab5b-a2f2fdf4453c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a6a587aedea4024aa5cf14dbe3f51e5)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:08.400972 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa3ef7c-2670-4b4c-ab5b-a2f2fdf4453c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:08.526058 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-bfa3ef7c-2670-4b4c-ab5b-a2f2fdf4453c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a6a587aedea4024aa5cf14dbe3f51e5', 'request_id': 'req-bfa3ef7c-2670-4b4c-ab5b-a2f2fdf4453c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:08.541606 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-bfa3ef7c-2670-4b4c-ab5b-a2f2fdf4453c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a6a587aedea4024aa5cf14dbe3f51e5', 'request_id': 'req-bfa3ef7c-2670-4b4c-ab5b-a2f2fdf4453c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:08.556744 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-72e7e85d-f591-40ae-963e-295c51c78108 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f8c3fa7783a444a9883efe4ec0c6d8d', 'request_id': 'req-72e7e85d-f591-40ae-963e-295c51c78108'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:08.564837 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-bfa3ef7c-2670-4b4c-ab5b-a2f2fdf4453c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a6a587aedea4024aa5cf14dbe3f51e5', 'request_id': 'req-bfa3ef7c-2670-4b4c-ab5b-a2f2fdf4453c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:08.571561 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-72e7e85d-f591-40ae-963e-295c51c78108 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f8c3fa7783a444a9883efe4ec0c6d8d', 'request_id': 'req-72e7e85d-f591-40ae-963e-295c51c78108'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:08.583032 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-72e7e85d-f591-40ae-963e-295c51c78108 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f8c3fa7783a444a9883efe4ec0c6d8d', 'request_id': 'req-72e7e85d-f591-40ae-963e-295c51c78108'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:08.602912 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-72e7e85d-f591-40ae-963e-295c51c78108 admin admin] The token cache is being invalidate because project 3f8c3fa7783a444a9883efe4ec0c6d8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:08.603217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-72e7e85d-f591-40ae-963e-295c51c78108 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:08.604399 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-72e7e85d-f591-40ae-963e-295c51c78108 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:08.604969 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-bfa3ef7c-2670-4b4c-ab5b-a2f2fdf4453c admin admin] The token cache is being invalidate because project 8a6a587aedea4024aa5cf14dbe3f51e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:08.609617 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 962/1924] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:08 2022] DELETE /identity/v3/projects/3f8c3fa7783a444a9883efe4ec0c6d8d => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:08.609794 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-bfa3ef7c-2670-4b4c-ab5b-a2f2fdf4453c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:08.609794 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-bfa3ef7c-2670-4b4c-ab5b-a2f2fdf4453c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:08.615291 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7e85d-f591-40ae-963e-295c51c78108 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:08.617325 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-72e7e85d-f591-40ae-963e-295c51c78108 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:08.623692 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 963/1925] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:08 2022] DELETE /identity/v3/projects/8a6a587aedea4024aa5cf14dbe3f51e5 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:08.625017 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa3ef7c-2670-4b4c-ab5b-a2f2fdf4453c admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:08.628785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-bfa3ef7c-2670-4b4c-ab5b-a2f2fdf4453c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:08.713436 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:08.746780 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa3ef7c-2670-4b4c-ab5b-a2f2fdf4453c admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:08.763068 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514bfb02-e1bc-4c6d-bbc8-3030f48d3fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:08.764052 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514bfb02-e1bc-4c6d-bbc8-3030f48d3fe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:08.764528 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514bfb02-e1bc-4c6d-bbc8-3030f48d3fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:08.764851 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7e85d-f591-40ae-963e-295c51c78108 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:08.765797 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514bfb02-e1bc-4c6d-bbc8-3030f48d3fe1 admin admin] PATH_INFO: `/v3/users/97c54b95585a4ffa93502ab65f6579b4` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:08.768220 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514bfb02-e1bc-4c6d-bbc8-3030f48d3fe1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97c54b95585a4ffa93502ab65f6579b4)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:08.770625 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab224d4-6263-48d0-ba63-c59dd8565b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:08.771482 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab224d4-6263-48d0-ba63-c59dd8565b6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:08.771822 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab224d4-6263-48d0-ba63-c59dd8565b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:08.772108 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab224d4-6263-48d0-ba63-c59dd8565b6f admin admin] PATH_INFO: `/v3/users/5c9aeef001374bd5a22a8a0fc8ca3f61` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:08.773821 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514bfb02-e1bc-4c6d-bbc8-3030f48d3fe1 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:08.774213 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab224d4-6263-48d0-ba63-c59dd8565b6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c9aeef001374bd5a22a8a0fc8ca3f61)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:08.777568 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab224d4-6263-48d0-ba63-c59dd8565b6f admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:08.865979 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:08.871472 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-cab224d4-6263-48d0-ba63-c59dd8565b6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c9aeef001374bd5a22a8a0fc8ca3f61'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:08.872593 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-cab224d4-6263-48d0-ba63-c59dd8565b6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c9aeef001374bd5a22a8a0fc8ca3f61'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:08.880230 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-514bfb02-e1bc-4c6d-bbc8-3030f48d3fe1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97c54b95585a4ffa93502ab65f6579b4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:08.881375 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-514bfb02-e1bc-4c6d-bbc8-3030f48d3fe1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97c54b95585a4ffa93502ab65f6579b4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:08.899297 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-cab224d4-6263-48d0-ba63-c59dd8565b6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c9aeef001374bd5a22a8a0fc8ca3f61'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:08.910608 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-514bfb02-e1bc-4c6d-bbc8-3030f48d3fe1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97c54b95585a4ffa93502ab65f6579b4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:08.918023 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-cab224d4-6263-48d0-ba63-c59dd8565b6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c9aeef001374bd5a22a8a0fc8ca3f61'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:08.921817 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-514bfb02-e1bc-4c6d-bbc8-3030f48d3fe1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97c54b95585a4ffa93502ab65f6579b4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:08.923221 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-514bfb02-e1bc-4c6d-bbc8-3030f48d3fe1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97c54b95585a4ffa93502ab65f6579b4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:08.938415 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-cab224d4-6263-48d0-ba63-c59dd8565b6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c9aeef001374bd5a22a8a0fc8ca3f61'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:08.945450 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 964/1926] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:08 2022] DELETE /identity/v3/users/97c54b95585a4ffa93502ab65f6579b4 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:08.951329 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 963/1927] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:08 2022] DELETE /identity/v3/users/5c9aeef001374bd5a22a8a0fc8ca3f61 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:09.819657 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514bfb02-e1bc-4c6d-bbc8-3030f48d3fe1 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:09.821245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-514bfb02-e1bc-4c6d-bbc8-3030f48d3fe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:09.827241 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab224d4-6263-48d0-ba63-c59dd8565b6f admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:09.829467 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-cab224d4-6263-48d0-ba63-c59dd8565b6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:09.930174 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514bfb02-e1bc-4c6d-bbc8-3030f48d3fe1 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:09.935011 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f55f8-f27d-47ba-a0bf-d576afc275e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:09.935713 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f55f8-f27d-47ba-a0bf-d576afc275e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:09.936003 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f55f8-f27d-47ba-a0bf-d576afc275e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:09.936321 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f55f8-f27d-47ba-a0bf-d576afc275e3 admin admin] PATH_INFO: `/v3/projects/25c402960b534b4f8c808a3e10637c3d` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:09.937499 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0f55f8-f27d-47ba-a0bf-d576afc275e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25c402960b534b4f8c808a3e10637c3d)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:09.939439 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0f55f8-f27d-47ba-a0bf-d576afc275e3 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:09.941695 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab224d4-6263-48d0-ba63-c59dd8565b6f admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:09.946682 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e97c0-3533-4edb-8510-4c9fc4716bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:09.947348 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e97c0-3533-4edb-8510-4c9fc4716bc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:09.947629 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e97c0-3533-4edb-8510-4c9fc4716bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:09.947904 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e97c0-3533-4edb-8510-4c9fc4716bc8 admin admin] PATH_INFO: `/v3/projects/1ae1a7528bfd4f348ddbad51a65a2604` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:09.949364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726e97c0-3533-4edb-8510-4c9fc4716bc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ae1a7528bfd4f348ddbad51a65a2604)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:09.951429 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726e97c0-3533-4edb-8510-4c9fc4716bc8 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:09.990854 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ad0f55f8-f27d-47ba-a0bf-d576afc275e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25c402960b534b4f8c808a3e10637c3d', 'request_id': 'req-ad0f55f8-f27d-47ba-a0bf-d576afc275e3'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:09.996625 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ad0f55f8-f27d-47ba-a0bf-d576afc275e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25c402960b534b4f8c808a3e10637c3d', 'request_id': 'req-ad0f55f8-f27d-47ba-a0bf-d576afc275e3'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:10.005700 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ad0f55f8-f27d-47ba-a0bf-d576afc275e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25c402960b534b4f8c808a3e10637c3d', 'request_id': 'req-ad0f55f8-f27d-47ba-a0bf-d576afc275e3'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:10.006096 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-726e97c0-3533-4edb-8510-4c9fc4716bc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ae1a7528bfd4f348ddbad51a65a2604', 'request_id': 'req-726e97c0-3533-4edb-8510-4c9fc4716bc8'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:10.007835 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:10.013850 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-726e97c0-3533-4edb-8510-4c9fc4716bc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ae1a7528bfd4f348ddbad51a65a2604', 'request_id': 'req-726e97c0-3533-4edb-8510-4c9fc4716bc8'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:10.019161 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ad0f55f8-f27d-47ba-a0bf-d576afc275e3 admin admin] The token cache is being invalidate because project 25c402960b534b4f8c808a3e10637c3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:10.020782 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ad0f55f8-f27d-47ba-a0bf-d576afc275e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:10.022524 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ad0f55f8-f27d-47ba-a0bf-d576afc275e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:10.024709 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-726e97c0-3533-4edb-8510-4c9fc4716bc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ae1a7528bfd4f348ddbad51a65a2604', 'request_id': 'req-726e97c0-3533-4edb-8510-4c9fc4716bc8'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:10.025239 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 965/1928] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:09 2022] DELETE /identity/v3/projects/25c402960b534b4f8c808a3e10637c3d => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:10.032499 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f55f8-f27d-47ba-a0bf-d576afc275e3 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:10.034447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ad0f55f8-f27d-47ba-a0bf-d576afc275e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:10.037877 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-726e97c0-3533-4edb-8510-4c9fc4716bc8 admin admin] The token cache is being invalidate because project 1ae1a7528bfd4f348ddbad51a65a2604 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:10.038173 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-726e97c0-3533-4edb-8510-4c9fc4716bc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:10.038876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-726e97c0-3533-4edb-8510-4c9fc4716bc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:10.044737 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 964/1929] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:09 2022] DELETE /identity/v3/projects/1ae1a7528bfd4f348ddbad51a65a2604 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:10.046147 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f30a8-a102-4cb8-b5b1-c047a30e4a39 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:10.046476 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f30a8-a102-4cb8-b5b1-c047a30e4a39 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:10.046790 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f30a8-a102-4cb8-b5b1-c047a30e4a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:10.068765 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-8c2f30a8-a102-4cb8-b5b1-c047a30e4a39 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:10.069943 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-8c2f30a8-a102-4cb8-b5b1-c047a30e4a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:10.110982 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f55f8-f27d-47ba-a0bf-d576afc275e3 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:10.118722 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:10.137471 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 965/1930] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:10.145422 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87da39f-4c25-4969-a143-fe0e4d86b8d2 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:10.145708 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87da39f-4c25-4969-a143-fe0e4d86b8d2 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:10.146020 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87da39f-4c25-4969-a143-fe0e4d86b8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:10.165965 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-a87da39f-4c25-4969-a143-fe0e4d86b8d2 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:10.167197 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a87da39f-4c25-4969-a143-fe0e4d86b8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:10.173398 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aebdf9-8314-4f6e-9ecd-99851a2b16ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:10.174105 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aebdf9-8314-4f6e-9ecd-99851a2b16ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:10.174391 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aebdf9-8314-4f6e-9ecd-99851a2b16ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:10.174700 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aebdf9-8314-4f6e-9ecd-99851a2b16ac admin admin] PATH_INFO: `/v3/users/c554168adba4462b94303f3a4318a119` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:10.175473 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 966/1931] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:10.175985 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aebdf9-8314-4f6e-9ecd-99851a2b16ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=c554168adba4462b94303f3a4318a119)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:10.178048 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aebdf9-8314-4f6e-9ecd-99851a2b16ac admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:10.180328 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87da39f-4c25-4969-a143-fe0e4d86b8d2 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:10.190143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87da39f-4c25-4969-a143-fe0e4d86b8d2 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:10.194928 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e8b1e2-2903-4572-befc-8ed3e63673d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:10.195564 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8b1e2-2903-4572-befc-8ed3e63673d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:10.195847 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8b1e2-2903-4572-befc-8ed3e63673d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:10.196124 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8b1e2-2903-4572-befc-8ed3e63673d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:10.196879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e8b1e2-2903-4572-befc-8ed3e63673d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:10.197192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e8b1e2-2903-4572-befc-8ed3e63673d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:10.198815 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e8b1e2-2903-4572-befc-8ed3e63673d3 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:10.211740 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 967/1932] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:09:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:09:10.219139 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e8b1e2-2903-4572-befc-8ed3e63673d3 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:10.228596 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e8b1e2-2903-4572-befc-8ed3e63673d3 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:10.234290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6db58c-c41e-4824-b7c0-c3565c1707b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:10.235043 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6db58c-c41e-4824-b7c0-c3565c1707b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:10.235697 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6db58c-c41e-4824-b7c0-c3565c1707b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:10.235987 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6db58c-c41e-4824-b7c0-c3565c1707b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:10.236885 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6db58c-c41e-4824-b7c0-c3565c1707b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:10.239131 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6db58c-c41e-4824-b7c0-c3565c1707b1 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:10.240636 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-95aebdf9-8314-4f6e-9ecd-99851a2b16ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c554168adba4462b94303f3a4318a119'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:10.241433 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-95aebdf9-8314-4f6e-9ecd-99851a2b16ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c554168adba4462b94303f3a4318a119'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:10.256756 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-95aebdf9-8314-4f6e-9ecd-99851a2b16ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c554168adba4462b94303f3a4318a119'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:10.265211 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-95aebdf9-8314-4f6e-9ecd-99851a2b16ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c554168adba4462b94303f3a4318a119'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:10.265994 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-95aebdf9-8314-4f6e-9ecd-99851a2b16ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c554168adba4462b94303f3a4318a119'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:10.268694 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 968/1933] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:10 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:09:10.273760 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6db58c-c41e-4824-b7c0-c3565c1707b1 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:10.275863 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-cc6db58c-c41e-4824-b7c0-c3565c1707b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:10.280899 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 966/1934] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:10 2022] DELETE /identity/v3/users/c554168adba4462b94303f3a4318a119 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:10.356509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6db58c-c41e-4824-b7c0-c3565c1707b1 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:10.432967 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e045d9-35ba-4863-9329-1aae77965178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:10.433632 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e045d9-35ba-4863-9329-1aae77965178 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:10.433948 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e045d9-35ba-4863-9329-1aae77965178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:10.434425 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e045d9-35ba-4863-9329-1aae77965178 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:10.435388 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e045d9-35ba-4863-9329-1aae77965178 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:10.437627 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e045d9-35ba-4863-9329-1aae77965178 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:10.476738 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 969/1935] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:10 2022] POST /identity/v3/users => generated 292 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:09:10.480671 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aebdf9-8314-4f6e-9ecd-99851a2b16ac admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:10.505565 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aebdf9-8314-4f6e-9ecd-99851a2b16ac admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:10.514895 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4894d-ca83-4fcd-9cd3-32257e7e9d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:10.515555 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4894d-ca83-4fcd-9cd3-32257e7e9d7b admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:10.515836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4894d-ca83-4fcd-9cd3-32257e7e9d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:10.516112 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4894d-ca83-4fcd-9cd3-32257e7e9d7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:10.520729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d4894d-ca83-4fcd-9cd3-32257e7e9d7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:10.522398 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d4894d-ca83-4fcd-9cd3-32257e7e9d7b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:10.536453 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 967/1936] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:10 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:10.549609 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e045d9-35ba-4863-9329-1aae77965178 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:10.562797 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e045d9-35ba-4863-9329-1aae77965178 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:10.568376 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb2ccd8-9510-4342-af25-09202e044322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:10.569127 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb2ccd8-9510-4342-af25-09202e044322 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:10.569577 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb2ccd8-9510-4342-af25-09202e044322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:10.569918 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb2ccd8-9510-4342-af25-09202e044322 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:10.570623 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb2ccd8-9510-4342-af25-09202e044322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:10.572293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb2ccd8-9510-4342-af25-09202e044322 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:10.583335 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 970/1937] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:10.587920 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4894d-ca83-4fcd-9cd3-32257e7e9d7b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:10.598339 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4894d-ca83-4fcd-9cd3-32257e7e9d7b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:10.603011 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099f3e8-25bc-4343-ae13-0fb464cc578c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:10.603684 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099f3e8-25bc-4343-ae13-0fb464cc578c admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:10.603958 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099f3e8-25bc-4343-ae13-0fb464cc578c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:10.604234 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099f3e8-25bc-4343-ae13-0fb464cc578c admin admin] PATH_INFO: `/v3/projects/1604c2b5535844e4941852756166417d/users/f803ea8909df41d3abd581a3b18edde7/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:10.613387 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:10.621589 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2099f3e8-25bc-4343-ae13-0fb464cc578c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1604c2b5535844e4941852756166417d, user_id=f803ea8909df41d3abd581a3b18edde7, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:10.624476 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2099f3e8-25bc-4343-ae13-0fb464cc578c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:10.641094 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 968/1938] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:10 2022] PUT /identity/v3/projects/1604c2b5535844e4941852756166417d/users/f803ea8909df41d3abd581a3b18edde7/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:10.651981 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb2ccd8-9510-4342-af25-09202e044322 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:10.654158 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-4eb2ccd8-9510-4342-af25-09202e044322 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:10.654609 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5a987-3f51-4936-9df5-c748f06a1adb None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:10.655014 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5a987-3f51-4936-9df5-c748f06a1adb None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:10.655390 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5a987-3f51-4936-9df5-c748f06a1adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:10.672915 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb2ccd8-9510-4342-af25-09202e044322 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:10.689200 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-33f5a987-3f51-4936-9df5-c748f06a1adb None None] MFA Rules not processed for user `f803ea8909df41d3abd581a3b18edde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:10.690471 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-33f5a987-3f51-4936-9df5-c748f06a1adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:10.721187 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:10.776316 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302e4a1-d88a-4ae5-afea-15e8ea6b28dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:10.777244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302e4a1-d88a-4ae5-afea-15e8ea6b28dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:10.777528 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302e4a1-d88a-4ae5-afea-15e8ea6b28dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:10.777803 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302e4a1-d88a-4ae5-afea-15e8ea6b28dd admin admin] PATH_INFO: `/v3/projects/9e9b669e93674c7e9d76ca3cb1bb08d6` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:10.779483 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8302e4a1-d88a-4ae5-afea-15e8ea6b28dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e9b669e93674c7e9d76ca3cb1bb08d6)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:10.781551 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8302e4a1-d88a-4ae5-afea-15e8ea6b28dd admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:10.781982 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 969/1939] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:10.787803 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5a987-3f51-4936-9df5-c748f06a1adb None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:10.800057 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5a987-3f51-4936-9df5-c748f06a1adb None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:10.810025 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6294451-fef2-456d-bfa5-f187db442fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:10.810654 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6294451-fef2-456d-bfa5-f187db442fdb admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:10.812065 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6294451-fef2-456d-bfa5-f187db442fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:10.812202 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6294451-fef2-456d-bfa5-f187db442fdb admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:10.812866 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6294451-fef2-456d-bfa5-f187db442fdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:10.814815 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6294451-fef2-456d-bfa5-f187db442fdb admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:10.849250 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 970/1940] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:10 2022] POST /identity/v3/projects => generated 381 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:09:10.853181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6294451-fef2-456d-bfa5-f187db442fdb admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:10.853604 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8302e4a1-d88a-4ae5-afea-15e8ea6b28dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e9b669e93674c7e9d76ca3cb1bb08d6', 'request_id': 'req-8302e4a1-d88a-4ae5-afea-15e8ea6b28dd'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:10.861895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8302e4a1-d88a-4ae5-afea-15e8ea6b28dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e9b669e93674c7e9d76ca3cb1bb08d6', 'request_id': 'req-8302e4a1-d88a-4ae5-afea-15e8ea6b28dd'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:10.871558 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8302e4a1-d88a-4ae5-afea-15e8ea6b28dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e9b669e93674c7e9d76ca3cb1bb08d6', 'request_id': 'req-8302e4a1-d88a-4ae5-afea-15e8ea6b28dd'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:10.890279 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8302e4a1-d88a-4ae5-afea-15e8ea6b28dd admin admin] The token cache is being invalidate because project 9e9b669e93674c7e9d76ca3cb1bb08d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:10.890575 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8302e4a1-d88a-4ae5-afea-15e8ea6b28dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:10.891281 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8302e4a1-d88a-4ae5-afea-15e8ea6b28dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:10.899451 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 971/1941] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:10 2022] DELETE /identity/v3/projects/9e9b669e93674c7e9d76ca3cb1bb08d6 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:10.901100 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9a4a7-1265-4335-846a-cb330f9a1d17 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:10.901386 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9a4a7-1265-4335-846a-cb330f9a1d17 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:10.904929 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9a4a7-1265-4335-846a-cb330f9a1d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:10.941098 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-81d9a4a7-1265-4335-846a-cb330f9a1d17 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:10.943562 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-81d9a4a7-1265-4335-846a-cb330f9a1d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:10.960461 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 972/1942] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:10.960715 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6294451-fef2-456d-bfa5-f187db442fdb admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:10.967189 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36e6ec-4141-4c3f-8c2a-b850d893b5aa None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:10.968338 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36e6ec-4141-4c3f-8c2a-b850d893b5aa None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:10.968338 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36e6ec-4141-4c3f-8c2a-b850d893b5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:10.973500 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc4c896-a4d0-45f8-8fd2-e391b471a246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:10.974148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4c896-a4d0-45f8-8fd2-e391b471a246 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:10.974424 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4c896-a4d0-45f8-8fd2-e391b471a246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:10.974723 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4c896-a4d0-45f8-8fd2-e391b471a246 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:10.975386 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc4c896-a4d0-45f8-8fd2-e391b471a246 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:10.977528 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc4c896-a4d0-45f8-8fd2-e391b471a246 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:11.024130 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 971/1943] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:10 2022] POST /identity/v3/users => generated 297 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:09:11.025821 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-0c36e6ec-4141-4c3f-8c2a-b850d893b5aa None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:11.027201 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0c36e6ec-4141-4c3f-8c2a-b850d893b5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:11.028880 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc4c896-a4d0-45f8-8fd2-e391b471a246 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:11.031046 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ebc4c896-a4d0-45f8-8fd2-e391b471a246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:11.037174 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 973/1944] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:11.041674 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c36e6ec-4141-4c3f-8c2a-b850d893b5aa None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:11.046178 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc4c896-a4d0-45f8-8fd2-e391b471a246 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:11.052232 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00292e6a-8c96-42d4-b8f3-2e964678bbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:11.052923 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00292e6a-8c96-42d4-b8f3-2e964678bbb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:11.053208 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00292e6a-8c96-42d4-b8f3-2e964678bbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:11.053492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00292e6a-8c96-42d4-b8f3-2e964678bbb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:11.054200 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00292e6a-8c96-42d4-b8f3-2e964678bbb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:11.055769 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c36e6ec-4141-4c3f-8c2a-b850d893b5aa None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:11.055976 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00292e6a-8c96-42d4-b8f3-2e964678bbb5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:11.063467 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ad390-9f3e-4551-8e69-59965cd5d51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:11.064847 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ad390-9f3e-4551-8e69-59965cd5d51e admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:11.065132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ad390-9f3e-4551-8e69-59965cd5d51e admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:11.065467 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ad390-9f3e-4551-8e69-59965cd5d51e admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:11.066212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ad390-9f3e-4551-8e69-59965cd5d51e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:11.066520 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ad390-9f3e-4551-8e69-59965cd5d51e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:11.068174 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ad390-9f3e-4551-8e69-59965cd5d51e admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:11.071495 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 972/1945] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:11.072577 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00292e6a-8c96-42d4-b8f3-2e964678bbb5 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:11.085196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 974/1946] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:09:11.086894 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00292e6a-8c96-42d4-b8f3-2e964678bbb5 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:11.089859 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ad390-9f3e-4551-8e69-59965cd5d51e admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:11.092917 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944fe543-83a3-4da5-b910-6bede30415e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:11.093828 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944fe543-83a3-4da5-b910-6bede30415e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:11.094299 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944fe543-83a3-4da5-b910-6bede30415e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:11.094722 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944fe543-83a3-4da5-b910-6bede30415e0 admin admin] PATH_INFO: `/v3/projects/cfb3ef0cc3b64226896e59ab6b2945c3/users/9d954655d5db411581a74a2d2ce10257/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:11.102231 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ad390-9f3e-4551-8e69-59965cd5d51e admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:11.107688 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd872e-5f7f-4d37-a9d6-b08689659c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:11.108377 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd872e-5f7f-4d37-a9d6-b08689659c33 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:11.108663 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd872e-5f7f-4d37-a9d6-b08689659c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:11.108940 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd872e-5f7f-4d37-a9d6-b08689659c33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:11.109632 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bd872e-5f7f-4d37-a9d6-b08689659c33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:11.111600 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bd872e-5f7f-4d37-a9d6-b08689659c33 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:11.120142 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944fe543-83a3-4da5-b910-6bede30415e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfb3ef0cc3b64226896e59ab6b2945c3, user_id=9d954655d5db411581a74a2d2ce10257, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:11.123889 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944fe543-83a3-4da5-b910-6bede30415e0 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:11.153542 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 973/1947] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:11 2022] PUT /identity/v3/projects/cfb3ef0cc3b64226896e59ab6b2945c3/users/9d954655d5db411581a74a2d2ce10257/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:11.153790 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 975/1948] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:11 2022] POST /identity/v3/projects => generated 367 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:09:11.155292 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944fe543-83a3-4da5-b910-6bede30415e0 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:11.158439 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd872e-5f7f-4d37-a9d6-b08689659c33 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:11.179492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:11.252696 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd872e-5f7f-4d37-a9d6-b08689659c33 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:11.257675 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76892f3e-0e46-45fa-a1e6-60b467998fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:11.258328 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76892f3e-0e46-45fa-a1e6-60b467998fa3 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:11.258609 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76892f3e-0e46-45fa-a1e6-60b467998fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:11.258885 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76892f3e-0e46-45fa-a1e6-60b467998fa3 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:11.259584 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76892f3e-0e46-45fa-a1e6-60b467998fa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:11.261653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76892f3e-0e46-45fa-a1e6-60b467998fa3 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:11.264993 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944fe543-83a3-4da5-b910-6bede30415e0 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:11.269999 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b493476-4af6-4883-9e1e-a12a25ec7339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:11.270659 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b493476-4af6-4883-9e1e-a12a25ec7339 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:11.270937 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b493476-4af6-4883-9e1e-a12a25ec7339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:11.271220 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b493476-4af6-4883-9e1e-a12a25ec7339 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:11.271934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b493476-4af6-4883-9e1e-a12a25ec7339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:11.273614 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b493476-4af6-4883-9e1e-a12a25ec7339 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:11.285946 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 974/1949] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:11.288514 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 976/1950] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:11 2022] POST /identity/v3/users => generated 284 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:09:11.292401 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b493476-4af6-4883-9e1e-a12a25ec7339 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:11.293586 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76892f3e-0e46-45fa-a1e6-60b467998fa3 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:11.309363 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76892f3e-0e46-45fa-a1e6-60b467998fa3 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:11.309607 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b493476-4af6-4883-9e1e-a12a25ec7339 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:11.316859 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc7587-c8b0-4641-842e-3b8fccc1e17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:11.317169 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0261e12e-3dfb-407a-a6ad-575e57611d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:11.317831 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc7587-c8b0-4641-842e-3b8fccc1e17d admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:11.317975 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261e12e-3dfb-407a-a6ad-575e57611d14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:11.318284 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261e12e-3dfb-407a-a6ad-575e57611d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:11.318408 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc7587-c8b0-4641-842e-3b8fccc1e17d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:11.318717 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261e12e-3dfb-407a-a6ad-575e57611d14 admin admin] PATH_INFO: `/v3/projects/cfb3ef0cc3b64226896e59ab6b2945c3/users/9d954655d5db411581a74a2d2ce10257/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:11.318856 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc7587-c8b0-4641-842e-3b8fccc1e17d admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:11.319887 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc7587-c8b0-4641-842e-3b8fccc1e17d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:11.322055 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0261e12e-3dfb-407a-a6ad-575e57611d14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfb3ef0cc3b64226896e59ab6b2945c3, user_id=9d954655d5db411581a74a2d2ce10257, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:11.322726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc7587-c8b0-4641-842e-3b8fccc1e17d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:11.325136 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:11.326633 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0261e12e-3dfb-407a-a6ad-575e57611d14 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:11.343829 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 977/1951] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:11.347332 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 975/1952] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:11 2022] PUT /identity/v3/projects/cfb3ef0cc3b64226896e59ab6b2945c3/users/9d954655d5db411581a74a2d2ce10257/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:11.357575 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc7587-c8b0-4641-842e-3b8fccc1e17d admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:11.358748 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908cfda-4d9b-4ae4-b2c3-dc2d454cfedb None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:11.359121 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908cfda-4d9b-4ae4-b2c3-dc2d454cfedb None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:11.359523 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908cfda-4d9b-4ae4-b2c3-dc2d454cfedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:11.372875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc7587-c8b0-4641-842e-3b8fccc1e17d admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:11.407342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-e908cfda-4d9b-4ae4-b2c3-dc2d454cfedb None None] MFA Rules not processed for user `9d954655d5db411581a74a2d2ce10257`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:11.408650 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e908cfda-4d9b-4ae4-b2c3-dc2d454cfedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:11.459274 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59bf62-8f4b-419d-95a4-3aaf6611c0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:11.460406 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59bf62-8f4b-419d-95a4-3aaf6611c0ab admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:11.460910 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59bf62-8f4b-419d-95a4-3aaf6611c0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:11.461404 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59bf62-8f4b-419d-95a4-3aaf6611c0ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:11.462352 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a59bf62-8f4b-419d-95a4-3aaf6611c0ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:11.463993 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a59bf62-8f4b-419d-95a4-3aaf6611c0ab admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:11.475439 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 978/1953] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:11.479970 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59bf62-8f4b-419d-95a4-3aaf6611c0ab admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:11.491531 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59bf62-8f4b-419d-95a4-3aaf6611c0ab admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:11.497508 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ebffc0-90d4-4b04-91f4-6c75794cdc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:11.498513 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebffc0-90d4-4b04-91f4-6c75794cdc43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:11.498901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebffc0-90d4-4b04-91f4-6c75794cdc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:11.499267 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebffc0-90d4-4b04-91f4-6c75794cdc43 admin admin] PATH_INFO: `/v3/projects/e4047e4eb89d47ddbe4c4091b610b6c6/users/ee51ee38a10146e39c9afbd3946454d9/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:11.508209 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 976/1954] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:11 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:11.520183 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e908cfda-4d9b-4ae4-b2c3-dc2d454cfedb None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:11.522551 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e908cfda-4d9b-4ae4-b2c3-dc2d454cfedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:11.524854 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ebffc0-90d4-4b04-91f4-6c75794cdc43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4047e4eb89d47ddbe4c4091b610b6c6, user_id=ee51ee38a10146e39c9afbd3946454d9, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:11.527617 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ebffc0-90d4-4b04-91f4-6c75794cdc43 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:11.548595 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 979/1955] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:11 2022] PUT /identity/v3/projects/e4047e4eb89d47ddbe4c4091b610b6c6/users/ee51ee38a10146e39c9afbd3946454d9/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:11.563751 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0ef48-e1da-4bed-8003-df742e424e03 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:11.564133 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0ef48-e1da-4bed-8003-df742e424e03 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:11.564523 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0ef48-e1da-4bed-8003-df742e424e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:11.597601 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-4dd0ef48-e1da-4bed-8003-df742e424e03 None None] MFA Rules not processed for user `ee51ee38a10146e39c9afbd3946454d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:11.599557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-4dd0ef48-e1da-4bed-8003-df742e424e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:11.635523 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e908cfda-4d9b-4ae4-b2c3-dc2d454cfedb None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:11.674574 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:11.693978 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 980/1956] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:11 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:11.698280 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5581e-2773-4d3b-9cda-573180bc5f6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:11.699064 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5581e-2773-4d3b-9cda-573180bc5f6a service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:11.699197 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0ef48-e1da-4bed-8003-df742e424e03 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:11.699350 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5581e-2773-4d3b-9cda-573180bc5f6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:11.699626 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5581e-2773-4d3b-9cda-573180bc5f6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:11.763007 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:11.767432 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c5581e-2773-4d3b-9cda-573180bc5f6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:11.768808 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0ef48-e1da-4bed-8003-df742e424e03 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:11.769031 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c5581e-2773-4d3b-9cda-573180bc5f6a service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:11.775141 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded6472-91dd-4af0-8c4f-dbc2ca04e384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:11.775786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded6472-91dd-4af0-8c4f-dbc2ca04e384 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:11.776070 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded6472-91dd-4af0-8c4f-dbc2ca04e384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:11.776384 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded6472-91dd-4af0-8c4f-dbc2ca04e384 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:11.777077 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eded6472-91dd-4af0-8c4f-dbc2ca04e384 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:11.779356 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eded6472-91dd-4af0-8c4f-dbc2ca04e384 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:11.780634 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 977/1957] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:11 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:09:11.810269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 981/1958] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:11 2022] POST /identity/v3/projects => generated 367 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:11.815744 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5581e-2773-4d3b-9cda-573180bc5f6a service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:11.830662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5581e-2773-4d3b-9cda-573180bc5f6a service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:11.906141 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381291b-92ab-4cc9-98bb-21e78ea5fbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:11.907020 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381291b-92ab-4cc9-98bb-21e78ea5fbd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:11.907296 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded6472-91dd-4af0-8c4f-dbc2ca04e384 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:11.907835 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381291b-92ab-4cc9-98bb-21e78ea5fbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:11.907835 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381291b-92ab-4cc9-98bb-21e78ea5fbd5 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:11.908568 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2381291b-92ab-4cc9-98bb-21e78ea5fbd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:11.910570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2381291b-92ab-4cc9-98bb-21e78ea5fbd5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:11.934436 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 978/1959] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:11 2022] POST /identity/v3/users => generated 290 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:11.938937 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381291b-92ab-4cc9-98bb-21e78ea5fbd5 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:11.950589 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381291b-92ab-4cc9-98bb-21e78ea5fbd5 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:11.956704 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed7ef7-07f3-4d43-b9f8-1a1fc90d0341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:11.957379 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed7ef7-07f3-4d43-b9f8-1a1fc90d0341 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:11.957662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed7ef7-07f3-4d43-b9f8-1a1fc90d0341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:11.957973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed7ef7-07f3-4d43-b9f8-1a1fc90d0341 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:11.959194 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ed7ef7-07f3-4d43-b9f8-1a1fc90d0341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:11.960944 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ed7ef7-07f3-4d43-b9f8-1a1fc90d0341 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:11.972795 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 979/1960] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:11.977751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed7ef7-07f3-4d43-b9f8-1a1fc90d0341 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:11.984100 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded6472-91dd-4af0-8c4f-dbc2ca04e384 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:11.988404 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbbffd4-0350-473e-a7df-60703f249f13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:11.989054 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbffd4-0350-473e-a7df-60703f249f13 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:11.989325 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbffd4-0350-473e-a7df-60703f249f13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:11.989637 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbffd4-0350-473e-a7df-60703f249f13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:11.990388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed7ef7-07f3-4d43-b9f8-1a1fc90d0341 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:11.991611 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1dbbffd4-0350-473e-a7df-60703f249f13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:11.995282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b0bfc-7bb2-48aa-a952-c83342d32178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:11.995982 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b0bfc-7bb2-48aa-a952-c83342d32178 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:11.996265 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b0bfc-7bb2-48aa-a952-c83342d32178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:11.996570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b0bfc-7bb2-48aa-a952-c83342d32178 admin admin] PATH_INFO: `/v3/projects/72623015993c45e9b1f31fa64dae8435/users/197932865c1a40679dd3f40ac54fe761/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:12.016799 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783b0bfc-7bb2-48aa-a952-c83342d32178 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72623015993c45e9b1f31fa64dae8435, user_id=197932865c1a40679dd3f40ac54fe761, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:12.019637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783b0bfc-7bb2-48aa-a952-c83342d32178 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:12.047016 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 980/1961] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:11 2022] PUT /identity/v3/projects/72623015993c45e9b1f31fa64dae8435/users/197932865c1a40679dd3f40ac54fe761/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:12.054253 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b0bfc-7bb2-48aa-a952-c83342d32178 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:12.086849 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbbffd4-0350-473e-a7df-60703f249f13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:12.088878 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbbffd4-0350-473e-a7df-60703f249f13 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:12.102471 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 982/1962] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:09:12.135245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:12.169826 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b0bfc-7bb2-48aa-a952-c83342d32178 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:12.176860 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81c9ca-9314-4c6b-8f1f-869a6309aa7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:12.177909 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81c9ca-9314-4c6b-8f1f-869a6309aa7c admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:12.178148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81c9ca-9314-4c6b-8f1f-869a6309aa7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:12.178447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81c9ca-9314-4c6b-8f1f-869a6309aa7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:12.179139 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e81c9ca-9314-4c6b-8f1f-869a6309aa7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:12.180849 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e81c9ca-9314-4c6b-8f1f-869a6309aa7c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:12.194911 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 981/1963] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:12.201716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbbffd4-0350-473e-a7df-60703f249f13 service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:12.215201 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbbffd4-0350-473e-a7df-60703f249f13 service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:12.221184 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd5b28-0143-42ce-8638-2d129dea693c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:12.222492 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd5b28-0143-42ce-8638-2d129dea693c admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:12.222889 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd5b28-0143-42ce-8638-2d129dea693c admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:12.223265 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd5b28-0143-42ce-8638-2d129dea693c admin admin] PATH_INFO: `/v3/projects/72623015993c45e9b1f31fa64dae8435/users/197932865c1a40679dd3f40ac54fe761/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:12.225474 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd5b28-0143-42ce-8638-2d129dea693c admin admin] RBAC: Authorizing `identity:create_grant(project_id=72623015993c45e9b1f31fa64dae8435, user_id=197932865c1a40679dd3f40ac54fe761, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:12.228554 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd5b28-0143-42ce-8638-2d129dea693c admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:12.242827 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 983/1964] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:12 2022] PUT /identity/v3/projects/72623015993c45e9b1f31fa64dae8435/users/197932865c1a40679dd3f40ac54fe761/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:12.260496 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acfc936-989a-461c-9382-c6a0c874668f None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:12.260939 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acfc936-989a-461c-9382-c6a0c874668f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:12.261353 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acfc936-989a-461c-9382-c6a0c874668f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:12.311205 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-3acfc936-989a-461c-9382-c6a0c874668f None None] MFA Rules not processed for user `197932865c1a40679dd3f40ac54fe761`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:12.314620 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3acfc936-989a-461c-9382-c6a0c874668f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:12.453945 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 982/1965] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:12 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:12.502200 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd5b28-0143-42ce-8638-2d129dea693c admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:12.606384 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd5b28-0143-42ce-8638-2d129dea693c admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:12.610348 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51215c3-3cb7-45a8-8540-e7b34ca84b13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:12.611000 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51215c3-3cb7-45a8-8540-e7b34ca84b13 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:12.611282 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51215c3-3cb7-45a8-8540-e7b34ca84b13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:12.611565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51215c3-3cb7-45a8-8540-e7b34ca84b13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:12.630590 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51215c3-3cb7-45a8-8540-e7b34ca84b13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:12.632059 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51215c3-3cb7-45a8-8540-e7b34ca84b13 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:12.645512 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 984/1966] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:12 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:12.761889 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acfc936-989a-461c-9382-c6a0c874668f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:12.763477 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3acfc936-989a-461c-9382-c6a0c874668f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:12.776555 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acfc936-989a-461c-9382-c6a0c874668f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:12.780843 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c3fe9-fed2-4196-9d34-c3b3bb7fbaa1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:12.781487 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c3fe9-fed2-4196-9d34-c3b3bb7fbaa1 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:12.781766 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c3fe9-fed2-4196-9d34-c3b3bb7fbaa1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:12.782035 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c3fe9-fed2-4196-9d34-c3b3bb7fbaa1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:12.865751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c3fe9-fed2-4196-9d34-c3b3bb7fbaa1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:12.867319 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c3fe9-fed2-4196-9d34-c3b3bb7fbaa1 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:12.877480 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 983/1967] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:12 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:16.219161 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51215c3-3cb7-45a8-8540-e7b34ca84b13 service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:16.220868 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b51215c3-3cb7-45a8-8540-e7b34ca84b13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:16.260500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:16.349059 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51215c3-3cb7-45a8-8540-e7b34ca84b13 service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:16.355665 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6502b-b31e-4e0b-bcca-09de175d9f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:16.357798 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6502b-b31e-4e0b-bcca-09de175d9f5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:16.358372 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6502b-b31e-4e0b-bcca-09de175d9f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:16.358603 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6502b-b31e-4e0b-bcca-09de175d9f5a admin admin] PATH_INFO: `/v3/users/b5297b694c9b414ab5947c5545f07596` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:16.360066 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd6502b-b31e-4e0b-bcca-09de175d9f5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5297b694c9b414ab5947c5545f07596)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:16.363076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd6502b-b31e-4e0b-bcca-09de175d9f5a admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:16.499664 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-9bd6502b-b31e-4e0b-bcca-09de175d9f5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5297b694c9b414ab5947c5545f07596'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:16.500700 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-9bd6502b-b31e-4e0b-bcca-09de175d9f5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5297b694c9b414ab5947c5545f07596'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:16.511692 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-9bd6502b-b31e-4e0b-bcca-09de175d9f5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5297b694c9b414ab5947c5545f07596'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:16.521798 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-9bd6502b-b31e-4e0b-bcca-09de175d9f5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5297b694c9b414ab5947c5545f07596'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:16.539362 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-9bd6502b-b31e-4e0b-bcca-09de175d9f5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5297b694c9b414ab5947c5545f07596'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:16.557309 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 985/1968] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:16 2022] DELETE /identity/v3/users/b5297b694c9b414ab5947c5545f07596 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:16.559147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3a5ac-8117-4a32-ad74-2af206be9b4f None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:16.559702 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3a5ac-8117-4a32-ad74-2af206be9b4f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:16.560602 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3a5ac-8117-4a32-ad74-2af206be9b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:16.590737 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-e2a3a5ac-8117-4a32-ad74-2af206be9b4f None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:16.591998 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e2a3a5ac-8117-4a32-ad74-2af206be9b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:16.678076 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 984/1969] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:16.684008 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:16.690120 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6502b-b31e-4e0b-bcca-09de175d9f5a admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:16.691725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-9bd6502b-b31e-4e0b-bcca-09de175d9f5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:16.766719 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6502b-b31e-4e0b-bcca-09de175d9f5a admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:16.772080 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3c637-41af-49cc-8c35-e55b2e84e805 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:16.772859 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3c637-41af-49cc-8c35-e55b2e84e805 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:16.773281 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3c637-41af-49cc-8c35-e55b2e84e805 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:16.773618 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3c637-41af-49cc-8c35-e55b2e84e805 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:16.788339 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f3c637-41af-49cc-8c35-e55b2e84e805 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:16.790503 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f3c637-41af-49cc-8c35-e55b2e84e805 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:16.823436 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 986/1970] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:17.205249 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a3a5ac-8117-4a32-ad74-2af206be9b4f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:17.209289 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e2a3a5ac-8117-4a32-ad74-2af206be9b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:17.236803 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a3a5ac-8117-4a32-ad74-2af206be9b4f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:17.261010 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a452485-02cf-4841-a3a0-e086e77c88ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:17.262089 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a452485-02cf-4841-a3a0-e086e77c88ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:17.262546 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a452485-02cf-4841-a3a0-e086e77c88ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:17.263024 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a452485-02cf-4841-a3a0-e086e77c88ec admin admin] PATH_INFO: `/v3/projects/9b6a56cd8c4a4b73bfbccb3d5f7f0ba4` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:17.269901 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a452485-02cf-4841-a3a0-e086e77c88ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b6a56cd8c4a4b73bfbccb3d5f7f0ba4)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:17.273420 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a452485-02cf-4841-a3a0-e086e77c88ec admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:17.430019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3c637-41af-49cc-8c35-e55b2e84e805 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:17.432499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-92f3c637-41af-49cc-8c35-e55b2e84e805 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:17.444408 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6a452485-02cf-4841-a3a0-e086e77c88ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b6a56cd8c4a4b73bfbccb3d5f7f0ba4', 'request_id': 'req-6a452485-02cf-4841-a3a0-e086e77c88ec'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:17.467105 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6a452485-02cf-4841-a3a0-e086e77c88ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b6a56cd8c4a4b73bfbccb3d5f7f0ba4', 'request_id': 'req-6a452485-02cf-4841-a3a0-e086e77c88ec'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:17.473458 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3c637-41af-49cc-8c35-e55b2e84e805 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:17.488148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6a452485-02cf-4841-a3a0-e086e77c88ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b6a56cd8c4a4b73bfbccb3d5f7f0ba4', 'request_id': 'req-6a452485-02cf-4841-a3a0-e086e77c88ec'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:17.528020 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6a452485-02cf-4841-a3a0-e086e77c88ec admin admin] The token cache is being invalidate because project 9b6a56cd8c4a4b73bfbccb3d5f7f0ba4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:17.528789 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6a452485-02cf-4841-a3a0-e086e77c88ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:17.530224 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6a452485-02cf-4841-a3a0-e086e77c88ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:17.531676 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 985/1971] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:17 2022] DELETE /identity/v3/projects/9b6a56cd8c4a4b73bfbccb3d5f7f0ba4 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:17.548558 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a452485-02cf-4841-a3a0-e086e77c88ec admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:17.553064 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6a452485-02cf-4841-a3a0-e086e77c88ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:17.608612 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:17.645111 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc19b62-d758-42bb-80bb-d366f9e88798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:17.649726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc19b62-d758-42bb-80bb-d366f9e88798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:17.650171 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc19b62-d758-42bb-80bb-d366f9e88798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:17.651751 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc19b62-d758-42bb-80bb-d366f9e88798 admin admin] PATH_INFO: `/v3/users/f803ea8909df41d3abd581a3b18edde7` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:17.656496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc19b62-d758-42bb-80bb-d366f9e88798 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f803ea8909df41d3abd581a3b18edde7)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:17.660259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc19b62-d758-42bb-80bb-d366f9e88798 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:17.684756 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a452485-02cf-4841-a3a0-e086e77c88ec admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:17.693218 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cebd40a-5e23-479f-8dc6-0e2eac7f27d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:17.694834 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebd40a-5e23-479f-8dc6-0e2eac7f27d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:17.695071 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebd40a-5e23-479f-8dc6-0e2eac7f27d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:17.695447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebd40a-5e23-479f-8dc6-0e2eac7f27d7 admin admin] PATH_INFO: `/v3/users/4e8e9f312c104af39cb20091fa4024ad` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:17.697739 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cebd40a-5e23-479f-8dc6-0e2eac7f27d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e8e9f312c104af39cb20091fa4024ad)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:17.703436 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cebd40a-5e23-479f-8dc6-0e2eac7f27d7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:17.778522 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-6dc19b62-d758-42bb-80bb-d366f9e88798 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f803ea8909df41d3abd581a3b18edde7'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:17.779834 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-6dc19b62-d758-42bb-80bb-d366f9e88798 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f803ea8909df41d3abd581a3b18edde7'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:17.795382 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4cebd40a-5e23-479f-8dc6-0e2eac7f27d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e8e9f312c104af39cb20091fa4024ad'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:17.796750 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4cebd40a-5e23-479f-8dc6-0e2eac7f27d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e8e9f312c104af39cb20091fa4024ad'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:17.804425 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-6dc19b62-d758-42bb-80bb-d366f9e88798 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f803ea8909df41d3abd581a3b18edde7'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:17.815764 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4cebd40a-5e23-479f-8dc6-0e2eac7f27d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e8e9f312c104af39cb20091fa4024ad'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:17.824340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-6dc19b62-d758-42bb-80bb-d366f9e88798 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f803ea8909df41d3abd581a3b18edde7'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:17.824963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4cebd40a-5e23-479f-8dc6-0e2eac7f27d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e8e9f312c104af39cb20091fa4024ad'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:17.825826 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4cebd40a-5e23-479f-8dc6-0e2eac7f27d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e8e9f312c104af39cb20091fa4024ad'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:17.842460 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:17.851341 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 986/1972] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:17 2022] DELETE /identity/v3/users/4e8e9f312c104af39cb20091fa4024ad => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:17.856841 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-6dc19b62-d758-42bb-80bb-d366f9e88798 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f803ea8909df41d3abd581a3b18edde7'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:17.876324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 987/1973] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:17 2022] DELETE /identity/v3/users/f803ea8909df41d3abd581a3b18edde7 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:17.878237 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07d5c8-8c6b-41b7-baa3-cdc163919141 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:17.878526 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07d5c8-8c6b-41b7-baa3-cdc163919141 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:17.878798 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07d5c8-8c6b-41b7-baa3-cdc163919141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:17.902315 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-fd07d5c8-8c6b-41b7-baa3-cdc163919141 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:17.903953 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fd07d5c8-8c6b-41b7-baa3-cdc163919141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:18.000848 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 987/1974] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:18.021527 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc19b62-d758-42bb-80bb-d366f9e88798 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:18.023910 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-6dc19b62-d758-42bb-80bb-d366f9e88798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:18.185353 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc19b62-d758-42bb-80bb-d366f9e88798 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:18.191353 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2654a3-e96c-4818-968d-47f4df285664 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:18.192191 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2654a3-e96c-4818-968d-47f4df285664 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:18.192561 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2654a3-e96c-4818-968d-47f4df285664 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:18.192910 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2654a3-e96c-4818-968d-47f4df285664 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:18.210823 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2654a3-e96c-4818-968d-47f4df285664 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:18.213998 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2654a3-e96c-4818-968d-47f4df285664 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:18.248634 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 988/1975] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:18.497583 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd07d5c8-8c6b-41b7-baa3-cdc163919141 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:18.501078 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fd07d5c8-8c6b-41b7-baa3-cdc163919141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:18.548988 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd07d5c8-8c6b-41b7-baa3-cdc163919141 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:18.565070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39aa8b8-15a5-4c83-9ec8-4f2e1854d008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:18.565500 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39aa8b8-15a5-4c83-9ec8-4f2e1854d008 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:18.565722 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39aa8b8-15a5-4c83-9ec8-4f2e1854d008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:18.566019 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39aa8b8-15a5-4c83-9ec8-4f2e1854d008 admin admin] PATH_INFO: `/v3/projects/3d66b31a768742318b4de6e0b3d74c4a` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:18.567488 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39aa8b8-15a5-4c83-9ec8-4f2e1854d008 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d66b31a768742318b4de6e0b3d74c4a)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:18.570560 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39aa8b8-15a5-4c83-9ec8-4f2e1854d008 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:18.667388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e39aa8b8-15a5-4c83-9ec8-4f2e1854d008 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d66b31a768742318b4de6e0b3d74c4a', 'request_id': 'req-e39aa8b8-15a5-4c83-9ec8-4f2e1854d008'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:18.683816 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e39aa8b8-15a5-4c83-9ec8-4f2e1854d008 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d66b31a768742318b4de6e0b3d74c4a', 'request_id': 'req-e39aa8b8-15a5-4c83-9ec8-4f2e1854d008'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:18.703173 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e39aa8b8-15a5-4c83-9ec8-4f2e1854d008 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d66b31a768742318b4de6e0b3d74c4a', 'request_id': 'req-e39aa8b8-15a5-4c83-9ec8-4f2e1854d008'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:18.736686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e39aa8b8-15a5-4c83-9ec8-4f2e1854d008 admin admin] The token cache is being invalidate because project 3d66b31a768742318b4de6e0b3d74c4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:18.738911 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e39aa8b8-15a5-4c83-9ec8-4f2e1854d008 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:18.739128 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e39aa8b8-15a5-4c83-9ec8-4f2e1854d008 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:18.744789 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 988/1976] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:18 2022] DELETE /identity/v3/projects/3d66b31a768742318b4de6e0b3d74c4a => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:18.758991 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28b746-451e-4924-8819-e193b7c6ff72 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:18.761643 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28b746-451e-4924-8819-e193b7c6ff72 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:18.761643 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28b746-451e-4924-8819-e193b7c6ff72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:18.825248 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-6b28b746-451e-4924-8819-e193b7c6ff72 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:18.827660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-6b28b746-451e-4924-8819-e193b7c6ff72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:18.880781 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39aa8b8-15a5-4c83-9ec8-4f2e1854d008 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:18.883451 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e39aa8b8-15a5-4c83-9ec8-4f2e1854d008 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:19.079216 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39aa8b8-15a5-4c83-9ec8-4f2e1854d008 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:19.082632 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 989/1977] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:19.097270 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434962c1-0938-4bce-81bd-0b2b5f6657e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:19.098157 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f82a4c-fd51-40c4-a44c-e6f778956580 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:19.098446 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434962c1-0938-4bce-81bd-0b2b5f6657e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:19.098584 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434962c1-0938-4bce-81bd-0b2b5f6657e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:19.098861 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434962c1-0938-4bce-81bd-0b2b5f6657e0 admin admin] PATH_INFO: `/v3/projects/1604c2b5535844e4941852756166417d` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:19.099679 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f82a4c-fd51-40c4-a44c-e6f778956580 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:19.100180 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f82a4c-fd51-40c4-a44c-e6f778956580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:19.101062 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434962c1-0938-4bce-81bd-0b2b5f6657e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1604c2b5535844e4941852756166417d)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:19.103044 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434962c1-0938-4bce-81bd-0b2b5f6657e0 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:19.179759 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-90f82a4c-fd51-40c4-a44c-e6f778956580 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:19.187848 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:19.189082 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-90f82a4c-fd51-40c4-a44c-e6f778956580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:19.210673 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:19.241367 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-434962c1-0938-4bce-81bd-0b2b5f6657e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1604c2b5535844e4941852756166417d', 'request_id': 'req-434962c1-0938-4bce-81bd-0b2b5f6657e0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:19.254838 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-434962c1-0938-4bce-81bd-0b2b5f6657e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1604c2b5535844e4941852756166417d', 'request_id': 'req-434962c1-0938-4bce-81bd-0b2b5f6657e0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:19.270619 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-434962c1-0938-4bce-81bd-0b2b5f6657e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1604c2b5535844e4941852756166417d', 'request_id': 'req-434962c1-0938-4bce-81bd-0b2b5f6657e0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:19.303139 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-434962c1-0938-4bce-81bd-0b2b5f6657e0 admin admin] The token cache is being invalidate because project 1604c2b5535844e4941852756166417d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:19.303139 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-434962c1-0938-4bce-81bd-0b2b5f6657e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:19.303139 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-434962c1-0938-4bce-81bd-0b2b5f6657e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:19.308886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 989/1978] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:18 2022] DELETE /identity/v3/projects/1604c2b5535844e4941852756166417d => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:19.308886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434962c1-0938-4bce-81bd-0b2b5f6657e0 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:19.310846 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-434962c1-0938-4bce-81bd-0b2b5f6657e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:19.580331 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434962c1-0938-4bce-81bd-0b2b5f6657e0 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:19.591829 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3204fb69-0150-4351-9f6b-9ee491ec5de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:19.593324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 990/1979] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 498 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:19.594008 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204fb69-0150-4351-9f6b-9ee491ec5de5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:19.594521 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204fb69-0150-4351-9f6b-9ee491ec5de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:19.594685 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f82a4c-fd51-40c4-a44c-e6f778956580 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:19.595028 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204fb69-0150-4351-9f6b-9ee491ec5de5 admin admin] PATH_INFO: `/v3/users/b0175d8bed8846c4a3350bc0023b5bb4` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:19.598570 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-90f82a4c-fd51-40c4-a44c-e6f778956580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:19.602842 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3204fb69-0150-4351-9f6b-9ee491ec5de5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0175d8bed8846c4a3350bc0023b5bb4)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:19.608495 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3204fb69-0150-4351-9f6b-9ee491ec5de5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:19.628616 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f82a4c-fd51-40c4-a44c-e6f778956580 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:19.643439 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a0aa7-5314-42f0-8609-23088254e1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:19.644854 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a0aa7-5314-42f0-8609-23088254e1f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:19.648322 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a0aa7-5314-42f0-8609-23088254e1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:19.649062 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a0aa7-5314-42f0-8609-23088254e1f1 admin admin] PATH_INFO: `/v3/users/9d954655d5db411581a74a2d2ce10257` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:19.656772 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a0aa7-5314-42f0-8609-23088254e1f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d954655d5db411581a74a2d2ce10257)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:19.665714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a0aa7-5314-42f0-8609-23088254e1f1 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:19.760761 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3204fb69-0150-4351-9f6b-9ee491ec5de5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0175d8bed8846c4a3350bc0023b5bb4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:19.762061 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3204fb69-0150-4351-9f6b-9ee491ec5de5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0175d8bed8846c4a3350bc0023b5bb4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:19.801571 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3204fb69-0150-4351-9f6b-9ee491ec5de5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0175d8bed8846c4a3350bc0023b5bb4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:19.818083 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3204fb69-0150-4351-9f6b-9ee491ec5de5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0175d8bed8846c4a3350bc0023b5bb4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:19.819593 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3204fb69-0150-4351-9f6b-9ee491ec5de5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0175d8bed8846c4a3350bc0023b5bb4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:19.846850 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ee8a0aa7-5314-42f0-8609-23088254e1f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d954655d5db411581a74a2d2ce10257'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:19.848213 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ee8a0aa7-5314-42f0-8609-23088254e1f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d954655d5db411581a74a2d2ce10257'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:19.882399 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ee8a0aa7-5314-42f0-8609-23088254e1f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d954655d5db411581a74a2d2ce10257'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:19.885584 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 990/1980] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:19 2022] DELETE /identity/v3/users/b0175d8bed8846c4a3350bc0023b5bb4 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:19.898790 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3204fb69-0150-4351-9f6b-9ee491ec5de5 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:19.904105 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ee8a0aa7-5314-42f0-8609-23088254e1f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d954655d5db411581a74a2d2ce10257'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:19.906629 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3204fb69-0150-4351-9f6b-9ee491ec5de5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:19.952089 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ee8a0aa7-5314-42f0-8609-23088254e1f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d954655d5db411581a74a2d2ce10257'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:19.970421 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 991/1981] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:19 2022] DELETE /identity/v3/users/9d954655d5db411581a74a2d2ce10257 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:19.994490 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c34e0-c7ac-48b4-91df-e9a0cff7e33c None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:19.994951 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c34e0-c7ac-48b4-91df-e9a0cff7e33c None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:19.995405 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c34e0-c7ac-48b4-91df-e9a0cff7e33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:20.095465 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:20.150473 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-181c34e0-c7ac-48b4-91df-e9a0cff7e33c None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:20.159267 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-181c34e0-c7ac-48b4-91df-e9a0cff7e33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:20.197616 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3204fb69-0150-4351-9f6b-9ee491ec5de5 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:20.445752 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 992/1982] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:20.452228 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da57ea1-cf61-4242-b362-0264c398ea79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:20.453403 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da57ea1-cf61-4242-b362-0264c398ea79 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:20.453944 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da57ea1-cf61-4242-b362-0264c398ea79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:20.454511 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da57ea1-cf61-4242-b362-0264c398ea79 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:20.456020 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da57ea1-cf61-4242-b362-0264c398ea79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:20.456631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da57ea1-cf61-4242-b362-0264c398ea79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:20.459954 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da57ea1-cf61-4242-b362-0264c398ea79 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:20.518305 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 991/1983] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:09:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 632 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:20.525065 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c34e0-c7ac-48b4-91df-e9a0cff7e33c None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:20.527263 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-181c34e0-c7ac-48b4-91df-e9a0cff7e33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:20.576614 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da57ea1-cf61-4242-b362-0264c398ea79 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:20.580601 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2da57ea1-cf61-4242-b362-0264c398ea79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:20.625619 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c34e0-c7ac-48b4-91df-e9a0cff7e33c None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:20.648518 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b4147f-8d39-4d63-9eb5-8996cb958458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:20.649844 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b4147f-8d39-4d63-9eb5-8996cb958458 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:20.650048 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b4147f-8d39-4d63-9eb5-8996cb958458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:20.650530 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b4147f-8d39-4d63-9eb5-8996cb958458 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:20.651821 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b4147f-8d39-4d63-9eb5-8996cb958458 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:20.657090 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b4147f-8d39-4d63-9eb5-8996cb958458 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:20.774360 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 993/1984] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:20 2022] POST /identity/v3/projects => generated 385 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:20.779835 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b4147f-8d39-4d63-9eb5-8996cb958458 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:20.787620 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:20.913853 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da57ea1-cf61-4242-b362-0264c398ea79 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:20.983919 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b4147f-8d39-4d63-9eb5-8996cb958458 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:21.002384 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8397f8-3f03-4812-9fe3-a8c46824e339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:21.003946 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8397f8-3f03-4812-9fe3-a8c46824e339 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:21.004458 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8397f8-3f03-4812-9fe3-a8c46824e339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:21.004813 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8397f8-3f03-4812-9fe3-a8c46824e339 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:21.025904 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8397f8-3f03-4812-9fe3-a8c46824e339 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:21.031300 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8397f8-3f03-4812-9fe3-a8c46824e339 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:21.111124 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 994/1985] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:20 2022] POST /identity/v3/users => generated 293 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:09:21.114880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8397f8-3f03-4812-9fe3-a8c46824e339 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:21.124795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1e8397f8-3f03-4812-9fe3-a8c46824e339 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:21.185025 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8397f8-3f03-4812-9fe3-a8c46824e339 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:21.200132 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e604a79a-299a-4ea9-bc3e-4f3df8be7f51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:21.200825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604a79a-299a-4ea9-bc3e-4f3df8be7f51 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:21.201108 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604a79a-299a-4ea9-bc3e-4f3df8be7f51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:21.201385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604a79a-299a-4ea9-bc3e-4f3df8be7f51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:21.212691 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8781d3d2-5619-484b-8a70-871970c9c2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:21.215911 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781d3d2-5619-484b-8a70-871970c9c2ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:21.216185 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781d3d2-5619-484b-8a70-871970c9c2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:21.216519 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781d3d2-5619-484b-8a70-871970c9c2ba admin admin] PATH_INFO: `/v3/projects/cfb3ef0cc3b64226896e59ab6b2945c3` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:21.226468 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8781d3d2-5619-484b-8a70-871970c9c2ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfb3ef0cc3b64226896e59ab6b2945c3)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:21.227686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e604a79a-299a-4ea9-bc3e-4f3df8be7f51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:21.231346 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e604a79a-299a-4ea9-bc3e-4f3df8be7f51 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:21.235510 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8781d3d2-5619-484b-8a70-871970c9c2ba admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:21.252723 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 992/1986] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 677 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:09:21.253968 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e604a79a-299a-4ea9-bc3e-4f3df8be7f51 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:21.265316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e604a79a-299a-4ea9-bc3e-4f3df8be7f51 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:21.271564 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e648c-0228-4e2f-ba86-7b61b8a63371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:21.272217 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e648c-0228-4e2f-ba86-7b61b8a63371 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:21.272542 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e648c-0228-4e2f-ba86-7b61b8a63371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:21.272820 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e648c-0228-4e2f-ba86-7b61b8a63371 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:21.273621 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122e648c-0228-4e2f-ba86-7b61b8a63371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:21.275709 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122e648c-0228-4e2f-ba86-7b61b8a63371 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:21.294252 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 993/1987] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:21.298937 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e648c-0228-4e2f-ba86-7b61b8a63371 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:21.317922 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e648c-0228-4e2f-ba86-7b61b8a63371 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:21.330616 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c3a97-0dfb-40a6-82bd-44504166d616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:21.331552 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c3a97-0dfb-40a6-82bd-44504166d616 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:21.331847 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c3a97-0dfb-40a6-82bd-44504166d616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:21.332134 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c3a97-0dfb-40a6-82bd-44504166d616 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:21.333103 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c3a97-0dfb-40a6-82bd-44504166d616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:21.335193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c3a97-0dfb-40a6-82bd-44504166d616 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:21.375009 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 994/1988] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:21.376629 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c3a97-0dfb-40a6-82bd-44504166d616 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:21.407257 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c3a97-0dfb-40a6-82bd-44504166d616 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:21.438490 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c4811-5974-4a10-8967-911591cef66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:21.445446 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c4811-5974-4a10-8967-911591cef66e admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:21.446562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c4811-5974-4a10-8967-911591cef66e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:21.447893 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c4811-5974-4a10-8967-911591cef66e admin admin] PATH_INFO: `/v3/projects/816cdf3ad1a74c339f888274f2817967/users/150bcbeac1904aefb689bd633a467ea0/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:21.474422 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8781d3d2-5619-484b-8a70-871970c9c2ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfb3ef0cc3b64226896e59ab6b2945c3', 'request_id': 'req-8781d3d2-5619-484b-8a70-871970c9c2ba'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:21.478592 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:21.495869 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5c4811-5974-4a10-8967-911591cef66e admin admin] RBAC: Authorizing `identity:create_grant(project_id=816cdf3ad1a74c339f888274f2817967, user_id=150bcbeac1904aefb689bd633a467ea0, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:21.503346 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5c4811-5974-4a10-8967-911591cef66e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:21.517512 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8781d3d2-5619-484b-8a70-871970c9c2ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfb3ef0cc3b64226896e59ab6b2945c3', 'request_id': 'req-8781d3d2-5619-484b-8a70-871970c9c2ba'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:21.553955 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8781d3d2-5619-484b-8a70-871970c9c2ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfb3ef0cc3b64226896e59ab6b2945c3', 'request_id': 'req-8781d3d2-5619-484b-8a70-871970c9c2ba'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:21.562294 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 995/1989] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:21 2022] PUT /identity/v3/projects/816cdf3ad1a74c339f888274f2817967/users/150bcbeac1904aefb689bd633a467ea0/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:21.581196 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eda00c-fed8-42c7-a07a-ca782dbd91dc None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:21.582645 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eda00c-fed8-42c7-a07a-ca782dbd91dc None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:21.583747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eda00c-fed8-42c7-a07a-ca782dbd91dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:21.596970 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8781d3d2-5619-484b-8a70-871970c9c2ba admin admin] The token cache is being invalidate because project cfb3ef0cc3b64226896e59ab6b2945c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:21.597449 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8781d3d2-5619-484b-8a70-871970c9c2ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:21.598570 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-8781d3d2-5619-484b-8a70-871970c9c2ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:21.606642 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 995/1990] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:21 2022] DELETE /identity/v3/projects/cfb3ef0cc3b64226896e59ab6b2945c3 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:21.614421 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05123b2c-b5f8-4f8f-9bc4-946a40536245 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:21.614935 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05123b2c-b5f8-4f8f-9bc4-946a40536245 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:21.615416 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05123b2c-b5f8-4f8f-9bc4-946a40536245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:21.691860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-05123b2c-b5f8-4f8f-9bc4-946a40536245 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:21.698721 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-05123b2c-b5f8-4f8f-9bc4-946a40536245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:21.743777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-76eda00c-fed8-42c7-a07a-ca782dbd91dc None None] MFA Rules not processed for user `150bcbeac1904aefb689bd633a467ea0`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:21.753044 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-76eda00c-fed8-42c7-a07a-ca782dbd91dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:21.865497 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 996/1991] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:21.885930 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff94d3-cfe5-419b-8d05-47200ec1a9df None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:21.886626 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff94d3-cfe5-419b-8d05-47200ec1a9df None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:21.889588 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff94d3-cfe5-419b-8d05-47200ec1a9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:21.915389 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 996/1992] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:21.935785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eda00c-fed8-42c7-a07a-ca782dbd91dc None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:21.939555 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-76eda00c-fed8-42c7-a07a-ca782dbd91dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:21.942498 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-00ff94d3-cfe5-419b-8d05-47200ec1a9df None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:21.943741 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-00ff94d3-cfe5-419b-8d05-47200ec1a9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:21.967177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 997/1993] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:21.976945 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff94d3-cfe5-419b-8d05-47200ec1a9df None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:21.992979 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff94d3-cfe5-419b-8d05-47200ec1a9df None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:22.056339 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbac31-a73e-4c72-b424-79f8e203ea4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:22.057462 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbac31-a73e-4c72-b424-79f8e203ea4b admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:22.057951 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbac31-a73e-4c72-b424-79f8e203ea4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:22.058426 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbac31-a73e-4c72-b424-79f8e203ea4b admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:22.061370 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dbac31-a73e-4c72-b424-79f8e203ea4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:22.061921 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dbac31-a73e-4c72-b424-79f8e203ea4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:22.072816 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dbac31-a73e-4c72-b424-79f8e203ea4b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:22.080449 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:22.112007 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 998/1994] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:09:22.113765 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbac31-a73e-4c72-b424-79f8e203ea4b admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:22.118419 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-62dbac31-a73e-4c72-b424-79f8e203ea4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:22.161062 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbac31-a73e-4c72-b424-79f8e203ea4b admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:22.175794 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eda00c-fed8-42c7-a07a-ca782dbd91dc None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:22.179176 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d9f6b-0da9-4f26-a41d-bbc4208f143d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:22.181354 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d9f6b-0da9-4f26-a41d-bbc4208f143d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:22.182628 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d9f6b-0da9-4f26-a41d-bbc4208f143d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:22.183502 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d9f6b-0da9-4f26-a41d-bbc4208f143d admin admin] PATH_INFO: `/v3/projects/34cf7d36f6ac4ad6bc926527920c32b6` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:22.187662 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d9f6b-0da9-4f26-a41d-bbc4208f143d admin admin] RBAC: Authorizing `identity:delete_project(project_id=34cf7d36f6ac4ad6bc926527920c32b6)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:22.188936 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e637e22-ad82-4709-aa92-e2dce1b0365a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:22.190016 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d9f6b-0da9-4f26-a41d-bbc4208f143d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:22.190308 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e637e22-ad82-4709-aa92-e2dce1b0365a service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:22.191572 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e637e22-ad82-4709-aa92-e2dce1b0365a service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:22.192436 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e637e22-ad82-4709-aa92-e2dce1b0365a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:22.212843 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e637e22-ad82-4709-aa92-e2dce1b0365a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:22.214550 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e637e22-ad82-4709-aa92-e2dce1b0365a service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:22.227002 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 997/1995] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:09:22.228059 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e637e22-ad82-4709-aa92-e2dce1b0365a service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:22.241218 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e637e22-ad82-4709-aa92-e2dce1b0365a service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:22.247685 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5c454-36c8-4073-94fb-d86fc9c94d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:22.248632 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5c454-36c8-4073-94fb-d86fc9c94d4c admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:22.249134 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5c454-36c8-4073-94fb-d86fc9c94d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:22.249400 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5c454-36c8-4073-94fb-d86fc9c94d4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:22.250267 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca5c454-36c8-4073-94fb-d86fc9c94d4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:22.252602 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca5c454-36c8-4073-94fb-d86fc9c94d4c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:22.298643 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 998/1996] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:22 2022] POST /identity/v3/projects => generated 375 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:22.299552 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5c454-36c8-4073-94fb-d86fc9c94d4c admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:22.307567 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5ca5c454-36c8-4073-94fb-d86fc9c94d4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:22.309502 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-094d9f6b-0da9-4f26-a41d-bbc4208f143d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34cf7d36f6ac4ad6bc926527920c32b6', 'request_id': 'req-094d9f6b-0da9-4f26-a41d-bbc4208f143d'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:22.332799 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-094d9f6b-0da9-4f26-a41d-bbc4208f143d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34cf7d36f6ac4ad6bc926527920c32b6', 'request_id': 'req-094d9f6b-0da9-4f26-a41d-bbc4208f143d'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:22.351992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-094d9f6b-0da9-4f26-a41d-bbc4208f143d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34cf7d36f6ac4ad6bc926527920c32b6', 'request_id': 'req-094d9f6b-0da9-4f26-a41d-bbc4208f143d'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:22.379139 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-094d9f6b-0da9-4f26-a41d-bbc4208f143d admin admin] The token cache is being invalidate because project 34cf7d36f6ac4ad6bc926527920c32b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:22.379679 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-094d9f6b-0da9-4f26-a41d-bbc4208f143d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:22.382144 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-094d9f6b-0da9-4f26-a41d-bbc4208f143d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:22.388140 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 999/1997] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:22 2022] DELETE /identity/v3/projects/34cf7d36f6ac4ad6bc926527920c32b6 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:22.388140 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d9f6b-0da9-4f26-a41d-bbc4208f143d admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:22.390531 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-094d9f6b-0da9-4f26-a41d-bbc4208f143d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:22.470295 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:22.477812 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5c454-36c8-4073-94fb-d86fc9c94d4c admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:22.488700 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013410dc-f776-4de5-a43e-fac2d4fef854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:22.489794 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013410dc-f776-4de5-a43e-fac2d4fef854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:22.490024 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013410dc-f776-4de5-a43e-fac2d4fef854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:22.490466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013410dc-f776-4de5-a43e-fac2d4fef854 admin admin] PATH_INFO: `/v3/users/150bcbeac1904aefb689bd633a467ea0` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:22.492205 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013410dc-f776-4de5-a43e-fac2d4fef854 admin admin] RBAC: Authorizing `identity:delete_user(user_id=150bcbeac1904aefb689bd633a467ea0)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:22.494622 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013410dc-f776-4de5-a43e-fac2d4fef854 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:22.586427 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d9f6b-0da9-4f26-a41d-bbc4208f143d admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:22.586734 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-013410dc-f776-4de5-a43e-fac2d4fef854 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '150bcbeac1904aefb689bd633a467ea0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:22.587705 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-013410dc-f776-4de5-a43e-fac2d4fef854 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '150bcbeac1904aefb689bd633a467ea0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:22.598808 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c405b-2e4f-4622-b1c7-1ab937ab63c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:22.600637 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c405b-2e4f-4622-b1c7-1ab937ab63c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:22.601588 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c405b-2e4f-4622-b1c7-1ab937ab63c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:22.602351 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c405b-2e4f-4622-b1c7-1ab937ab63c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:22.604039 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c405b-2e4f-4622-b1c7-1ab937ab63c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:22.609932 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c405b-2e4f-4622-b1c7-1ab937ab63c9 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:22.620399 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-013410dc-f776-4de5-a43e-fac2d4fef854 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '150bcbeac1904aefb689bd633a467ea0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:22.639756 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-013410dc-f776-4de5-a43e-fac2d4fef854 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '150bcbeac1904aefb689bd633a467ea0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:22.641075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-013410dc-f776-4de5-a43e-fac2d4fef854 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '150bcbeac1904aefb689bd633a467ea0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:22.671606 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1000/1998] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:22 2022] POST /identity/v3/users => generated 288 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:22.674166 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c405b-2e4f-4622-b1c7-1ab937ab63c9 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:22.675356 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 999/1999] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:22 2022] DELETE /identity/v3/users/150bcbeac1904aefb689bd633a467ea0 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:22.678376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013410dc-f776-4de5-a43e-fac2d4fef854 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:22.680472 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b50c405b-2e4f-4622-b1c7-1ab937ab63c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:22.681073 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-013410dc-f776-4de5-a43e-fac2d4fef854 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:22.749436 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c405b-2e4f-4622-b1c7-1ab937ab63c9 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:22.803858 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013410dc-f776-4de5-a43e-fac2d4fef854 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:22.811283 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041928b-8e34-48f5-a10f-83239c52d265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:22.813413 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041928b-8e34-48f5-a10f-83239c52d265 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:22.813821 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041928b-8e34-48f5-a10f-83239c52d265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:22.814236 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041928b-8e34-48f5-a10f-83239c52d265 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:22.815154 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041928b-8e34-48f5-a10f-83239c52d265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:22.820832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041928b-8e34-48f5-a10f-83239c52d265 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:22.849244 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1000/2000] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:22.850802 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c472a92-71f2-47fd-9741-cc345bbd6f0f None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:22.851251 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c472a92-71f2-47fd-9741-cc345bbd6f0f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:22.851453 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c472a92-71f2-47fd-9741-cc345bbd6f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:22.908864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-2c472a92-71f2-47fd-9741-cc345bbd6f0f None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:22.910557 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2c472a92-71f2-47fd-9741-cc345bbd6f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:22.930505 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1001/2001] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:22.931934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c472a92-71f2-47fd-9741-cc345bbd6f0f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:23.006030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c472a92-71f2-47fd-9741-cc345bbd6f0f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:23.008991 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be937b-8509-4889-8c6b-85580fb827d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:23.010478 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be937b-8509-4889-8c6b-85580fb827d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:23.011183 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be937b-8509-4889-8c6b-85580fb827d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:23.012125 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be937b-8509-4889-8c6b-85580fb827d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:23.014053 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4be937b-8509-4889-8c6b-85580fb827d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:23.021836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4be937b-8509-4889-8c6b-85580fb827d9 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:23.036319 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ac09bd-a69a-4faa-8480-6d988543b4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:23.036319 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac09bd-a69a-4faa-8480-6d988543b4b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:23.036319 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac09bd-a69a-4faa-8480-6d988543b4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:23.036319 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac09bd-a69a-4faa-8480-6d988543b4b8 admin admin] PATH_INFO: `/v3/users/b911b435ebcf4f2b83cfd7b68589d73b` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:23.038210 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ac09bd-a69a-4faa-8480-6d988543b4b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b911b435ebcf4f2b83cfd7b68589d73b)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:23.043122 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ac09bd-a69a-4faa-8480-6d988543b4b8 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:23.071478 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1002/2002] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:23.071898 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be937b-8509-4889-8c6b-85580fb827d9 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:23.077403 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:23.082928 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d4be937b-8509-4889-8c6b-85580fb827d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:23.194653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-06ac09bd-a69a-4faa-8480-6d988543b4b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b911b435ebcf4f2b83cfd7b68589d73b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:23.196160 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-06ac09bd-a69a-4faa-8480-6d988543b4b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b911b435ebcf4f2b83cfd7b68589d73b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:23.209530 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-06ac09bd-a69a-4faa-8480-6d988543b4b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b911b435ebcf4f2b83cfd7b68589d73b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:23.224479 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-06ac09bd-a69a-4faa-8480-6d988543b4b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b911b435ebcf4f2b83cfd7b68589d73b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:23.245829 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-06ac09bd-a69a-4faa-8480-6d988543b4b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b911b435ebcf4f2b83cfd7b68589d73b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:23.256206 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1001/2003] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:22 2022] DELETE /identity/v3/users/b911b435ebcf4f2b83cfd7b68589d73b => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:23.257812 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ac09bd-a69a-4faa-8480-6d988543b4b8 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:23.261130 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-06ac09bd-a69a-4faa-8480-6d988543b4b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:23.328136 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be937b-8509-4889-8c6b-85580fb827d9 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:23.362653 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:23.409761 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ac09bd-a69a-4faa-8480-6d988543b4b8 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:23.418563 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f71c4b1-770e-4089-81ca-a1739517dcfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:23.419502 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f71c4b1-770e-4089-81ca-a1739517dcfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:23.419801 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f71c4b1-770e-4089-81ca-a1739517dcfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:23.421810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f71c4b1-770e-4089-81ca-a1739517dcfe admin admin] PATH_INFO: `/v3/projects/aa24f81f21d44d1d8980883b9576bde7/users/60eee823638646488d84e78b28265053/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:23.448382 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f71c4b1-770e-4089-81ca-a1739517dcfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa24f81f21d44d1d8980883b9576bde7, user_id=60eee823638646488d84e78b28265053, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:23.451798 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f71c4b1-770e-4089-81ca-a1739517dcfe admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:23.467162 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba2fc9-645d-48a1-8693-3ebccecbea87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:23.468487 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba2fc9-645d-48a1-8693-3ebccecbea87 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:23.469127 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba2fc9-645d-48a1-8693-3ebccecbea87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:23.469732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba2fc9-645d-48a1-8693-3ebccecbea87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:23.473421 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1002/2004] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:23 2022] PUT /identity/v3/projects/aa24f81f21d44d1d8980883b9576bde7/users/60eee823638646488d84e78b28265053/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:23.474479 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f71c4b1-770e-4089-81ca-a1739517dcfe admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:23.476601 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0f71c4b1-770e-4089-81ca-a1739517dcfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:23.489702 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f6ba2fc9-645d-48a1-8693-3ebccecbea87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:23.509119 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ba2fc9-645d-48a1-8693-3ebccecbea87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:23.513224 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ba2fc9-645d-48a1-8693-3ebccecbea87 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:23.559153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1003/2005] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:23.559153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c864c-0a90-407e-a511-69f84fcf75ec None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:23.559153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c864c-0a90-407e-a511-69f84fcf75ec None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:23.559153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c864c-0a90-407e-a511-69f84fcf75ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:23.642086 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-2a0c864c-0a90-407e-a511-69f84fcf75ec None None] MFA Rules not processed for user `60eee823638646488d84e78b28265053`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:23.644778 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2a0c864c-0a90-407e-a511-69f84fcf75ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:23.661859 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f71c4b1-770e-4089-81ca-a1739517dcfe admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:23.670955 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9186852-6316-4ef1-95ec-b7bedcc7740c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:23.671830 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9186852-6316-4ef1-95ec-b7bedcc7740c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:23.672403 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9186852-6316-4ef1-95ec-b7bedcc7740c admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:23.672861 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9186852-6316-4ef1-95ec-b7bedcc7740c admin admin] PATH_INFO: `/v3/users/ee51ee38a10146e39c9afbd3946454d9` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:23.675079 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9186852-6316-4ef1-95ec-b7bedcc7740c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee51ee38a10146e39c9afbd3946454d9)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:23.677946 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9186852-6316-4ef1-95ec-b7bedcc7740c admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:23.712125 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:23.804788 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e9186852-6316-4ef1-95ec-b7bedcc7740c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee51ee38a10146e39c9afbd3946454d9'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:23.806332 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e9186852-6316-4ef1-95ec-b7bedcc7740c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee51ee38a10146e39c9afbd3946454d9'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:23.819422 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1004/2006] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:23.826383 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c864c-0a90-407e-a511-69f84fcf75ec None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:23.827963 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e9186852-6316-4ef1-95ec-b7bedcc7740c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee51ee38a10146e39c9afbd3946454d9'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:23.847315 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e9186852-6316-4ef1-95ec-b7bedcc7740c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee51ee38a10146e39c9afbd3946454d9'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:23.847583 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c864c-0a90-407e-a511-69f84fcf75ec None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:23.856066 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa62d96-146e-4e50-8e58-39b2e878d03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:23.857210 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa62d96-146e-4e50-8e58-39b2e878d03c admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:23.857683 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa62d96-146e-4e50-8e58-39b2e878d03c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:23.858154 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa62d96-146e-4e50-8e58-39b2e878d03c admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:23.859463 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa62d96-146e-4e50-8e58-39b2e878d03c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:23.863422 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa62d96-146e-4e50-8e58-39b2e878d03c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:23.906121 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e9186852-6316-4ef1-95ec-b7bedcc7740c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee51ee38a10146e39c9afbd3946454d9'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:23.927167 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1005/2007] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:23 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:23.930820 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1003/2008] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:23 2022] DELETE /identity/v3/users/ee51ee38a10146e39c9afbd3946454d9 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:23.935536 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa62d96-146e-4e50-8e58-39b2e878d03c admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:23.940483 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a437f3-4c52-427a-bf55-0867a91fb020 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:23.940987 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a437f3-4c52-427a-bf55-0867a91fb020 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:23.941443 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a437f3-4c52-427a-bf55-0867a91fb020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:23.942321 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9aa62d96-146e-4e50-8e58-39b2e878d03c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:24.006099 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-f3a437f3-4c52-427a-bf55-0867a91fb020 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:24.007818 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-f3a437f3-4c52-427a-bf55-0867a91fb020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:24.162580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa62d96-146e-4e50-8e58-39b2e878d03c admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:24.280112 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:24.355270 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1004/2009] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:24.370543 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8c311-5ac3-4760-b87f-2a73c49035c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:24.370543 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8c311-5ac3-4760-b87f-2a73c49035c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:24.371147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8c311-5ac3-4760-b87f-2a73c49035c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:24.371609 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8c311-5ac3-4760-b87f-2a73c49035c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:24.372620 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a8c311-5ac3-4760-b87f-2a73c49035c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:24.376225 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a8c311-5ac3-4760-b87f-2a73c49035c3 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:24.386049 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a437f3-4c52-427a-bf55-0867a91fb020 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:24.389435 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-f3a437f3-4c52-427a-bf55-0867a91fb020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:24.435134 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1006/2010] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:23 2022] POST /identity/v3/users => generated 289 bytes in 507 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:24.436469 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8c311-5ac3-4760-b87f-2a73c49035c3 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:24.463453 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8c311-5ac3-4760-b87f-2a73c49035c3 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:24.479564 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c54c8-8c10-484b-84cd-afe3103dfe9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:24.479564 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c54c8-8c10-484b-84cd-afe3103dfe9d admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:24.479564 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c54c8-8c10-484b-84cd-afe3103dfe9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:24.479564 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c54c8-8c10-484b-84cd-afe3103dfe9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:24.480516 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c54c8-8c10-484b-84cd-afe3103dfe9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:24.481136 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c54c8-8c10-484b-84cd-afe3103dfe9d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:24.531330 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1007/2011] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:24.537671 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c54c8-8c10-484b-84cd-afe3103dfe9d admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:24.594973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c54c8-8c10-484b-84cd-afe3103dfe9d admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:24.631867 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500af2d8-9e6f-4ec7-bdbe-7a4ca23967a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:24.634331 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500af2d8-9e6f-4ec7-bdbe-7a4ca23967a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:24.635269 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500af2d8-9e6f-4ec7-bdbe-7a4ca23967a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:24.635960 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500af2d8-9e6f-4ec7-bdbe-7a4ca23967a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:24.640184 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500af2d8-9e6f-4ec7-bdbe-7a4ca23967a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:24.644862 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500af2d8-9e6f-4ec7-bdbe-7a4ca23967a2 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:24.656459 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a437f3-4c52-427a-bf55-0867a91fb020 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:24.663537 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a7c16-9efe-4c80-a481-728230609d6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:24.668892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a7c16-9efe-4c80-a481-728230609d6f service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:24.669499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a7c16-9efe-4c80-a481-728230609d6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:24.669943 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a7c16-9efe-4c80-a481-728230609d6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:24.682309 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1008/2012] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:24.685889 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500af2d8-9e6f-4ec7-bdbe-7a4ca23967a2 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:24.687958 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30a7c16-9efe-4c80-a481-728230609d6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:24.688932 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-500af2d8-9e6f-4ec7-bdbe-7a4ca23967a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:24.690679 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30a7c16-9efe-4c80-a481-728230609d6f service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:24.732828 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1005/2013] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:09:24.734880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a7c16-9efe-4c80-a481-728230609d6f service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:24.758827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500af2d8-9e6f-4ec7-bdbe-7a4ca23967a2 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:24.780499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a7c16-9efe-4c80-a481-728230609d6f service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:24.792859 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3898717a-1b51-483e-8bbe-a790ff20a511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:24.794353 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898717a-1b51-483e-8bbe-a790ff20a511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:24.794867 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898717a-1b51-483e-8bbe-a790ff20a511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:24.795359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898717a-1b51-483e-8bbe-a790ff20a511 admin admin] PATH_INFO: `/v3/projects/816cdf3ad1a74c339f888274f2817967` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:24.808244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dff427-d4b7-48c1-ae3b-0549995f548e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:24.813938 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dff427-d4b7-48c1-ae3b-0549995f548e admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:24.818617 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dff427-d4b7-48c1-ae3b-0549995f548e admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:24.820729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dff427-d4b7-48c1-ae3b-0549995f548e admin admin] PATH_INFO: `/v3/projects/0426318b15c54e28a94aa100c0035118/users/4ab6f9e242f049da8b768b55aea8ca96/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:24.824187 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3898717a-1b51-483e-8bbe-a790ff20a511 admin admin] RBAC: Authorizing `identity:delete_project(project_id=816cdf3ad1a74c339f888274f2817967)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:24.843903 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3898717a-1b51-483e-8bbe-a790ff20a511 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:24.918275 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dff427-d4b7-48c1-ae3b-0549995f548e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0426318b15c54e28a94aa100c0035118, user_id=4ab6f9e242f049da8b768b55aea8ca96, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:24.927989 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dff427-d4b7-48c1-ae3b-0549995f548e admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:24.976201 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1006/2014] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:24 2022] PUT /identity/v3/projects/0426318b15c54e28a94aa100c0035118/users/4ab6f9e242f049da8b768b55aea8ca96/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:24.992796 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3898717a-1b51-483e-8bbe-a790ff20a511 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '816cdf3ad1a74c339f888274f2817967', 'request_id': 'req-3898717a-1b51-483e-8bbe-a790ff20a511'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:24.993568 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dff427-d4b7-48c1-ae3b-0549995f548e admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:24.995502 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3898717a-1b51-483e-8bbe-a790ff20a511 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '816cdf3ad1a74c339f888274f2817967', 'request_id': 'req-3898717a-1b51-483e-8bbe-a790ff20a511'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:24.998168 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c9dff427-d4b7-48c1-ae3b-0549995f548e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:25.023714 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3898717a-1b51-483e-8bbe-a790ff20a511 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '816cdf3ad1a74c339f888274f2817967', 'request_id': 'req-3898717a-1b51-483e-8bbe-a790ff20a511'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:25.038101 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:25.045633 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3898717a-1b51-483e-8bbe-a790ff20a511 admin admin] The token cache is being invalidate because project 816cdf3ad1a74c339f888274f2817967 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:25.046093 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3898717a-1b51-483e-8bbe-a790ff20a511 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:25.048890 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3898717a-1b51-483e-8bbe-a790ff20a511 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:25.054776 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1009/2015] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:24 2022] DELETE /identity/v3/projects/816cdf3ad1a74c339f888274f2817967 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:25.056066 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae1249-2c55-4ad9-94b0-94ef7649ac9e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:25.056403 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae1249-2c55-4ad9-94b0-94ef7649ac9e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:25.056796 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae1249-2c55-4ad9-94b0-94ef7649ac9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:25.128378 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-21ae1249-2c55-4ad9-94b0-94ef7649ac9e None None] MFA Rules not processed for user `4ab6f9e242f049da8b768b55aea8ca96`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:25.130074 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-21ae1249-2c55-4ad9-94b0-94ef7649ac9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:25.352350 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1010/2016] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:25.352350 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a062698-3dcf-4060-bd46-1b5ea42c6f58 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:25.352350 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a062698-3dcf-4060-bd46-1b5ea42c6f58 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:25.352951 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a062698-3dcf-4060-bd46-1b5ea42c6f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:25.365326 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dff427-d4b7-48c1-ae3b-0549995f548e admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:25.389651 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b5d48-614f-4538-9993-1762d7a673d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:25.390846 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b5d48-614f-4538-9993-1762d7a673d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:25.391347 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b5d48-614f-4538-9993-1762d7a673d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:25.391853 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b5d48-614f-4538-9993-1762d7a673d3 admin admin] PATH_INFO: `/v3/projects/f9a2e02f2cff49258df6de06399f8239` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:25.398389 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b5d48-614f-4538-9993-1762d7a673d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9a2e02f2cff49258df6de06399f8239)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:25.406212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b5d48-614f-4538-9993-1762d7a673d3 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:25.417056 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8a062698-3dcf-4060-bd46-1b5ea42c6f58 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:25.426092 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8a062698-3dcf-4060-bd46-1b5ea42c6f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:25.462430 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1011/2017] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:25.467144 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a062698-3dcf-4060-bd46-1b5ea42c6f58 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:25.474970 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8a062698-3dcf-4060-bd46-1b5ea42c6f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:25.501379 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:25.519459 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a062698-3dcf-4060-bd46-1b5ea42c6f58 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:25.559354 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eee0a0-73b6-456c-abdc-77a65c5dd550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:25.560227 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eee0a0-73b6-456c-abdc-77a65c5dd550 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:25.560705 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eee0a0-73b6-456c-abdc-77a65c5dd550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:25.561153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eee0a0-73b6-456c-abdc-77a65c5dd550 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:25.562242 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eee0a0-73b6-456c-abdc-77a65c5dd550 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:25.565751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eee0a0-73b6-456c-abdc-77a65c5dd550 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:25.597774 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-7b6b5d48-614f-4538-9993-1762d7a673d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9a2e02f2cff49258df6de06399f8239', 'request_id': 'req-7b6b5d48-614f-4538-9993-1762d7a673d3'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:25.611815 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-7b6b5d48-614f-4538-9993-1762d7a673d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9a2e02f2cff49258df6de06399f8239', 'request_id': 'req-7b6b5d48-614f-4538-9993-1762d7a673d3'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:25.633243 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-7b6b5d48-614f-4538-9993-1762d7a673d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9a2e02f2cff49258df6de06399f8239', 'request_id': 'req-7b6b5d48-614f-4538-9993-1762d7a673d3'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:25.670724 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-7b6b5d48-614f-4538-9993-1762d7a673d3 admin admin] The token cache is being invalidate because project f9a2e02f2cff49258df6de06399f8239 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:25.671760 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-7b6b5d48-614f-4538-9993-1762d7a673d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:25.672934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-7b6b5d48-614f-4538-9993-1762d7a673d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:25.675859 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1007/2018] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:24 2022] DELETE /identity/v3/projects/f9a2e02f2cff49258df6de06399f8239 => generated 0 bytes in 699 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:25.677886 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4e623-bfb8-4f39-b393-772e3fda54ea None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:25.678578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4e623-bfb8-4f39-b393-772e3fda54ea None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:25.679114 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4e623-bfb8-4f39-b393-772e3fda54ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:25.690907 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1012/2019] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:25 2022] POST /identity/v3/projects => generated 375 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:25.690907 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93604a-212e-4c71-a0d8-44e34a722832 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:25.690907 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93604a-212e-4c71-a0d8-44e34a722832 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:25.694854 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93604a-212e-4c71-a0d8-44e34a722832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:25.715306 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-81f4e623-bfb8-4f39-b393-772e3fda54ea None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:25.717423 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-81f4e623-bfb8-4f39-b393-772e3fda54ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:25.771465 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-fd93604a-212e-4c71-a0d8-44e34a722832 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:25.777658 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fd93604a-212e-4c71-a0d8-44e34a722832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:25.811811 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1008/2020] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:25.813629 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4e623-bfb8-4f39-b393-772e3fda54ea None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:25.816197 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-81f4e623-bfb8-4f39-b393-772e3fda54ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:25.841225 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4e623-bfb8-4f39-b393-772e3fda54ea None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:25.858554 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115374f-232d-4099-9f0c-7f4bacc8d4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:25.860434 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115374f-232d-4099-9f0c-7f4bacc8d4c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:25.862309 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115374f-232d-4099-9f0c-7f4bacc8d4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:25.863118 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115374f-232d-4099-9f0c-7f4bacc8d4c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:25.864896 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115374f-232d-4099-9f0c-7f4bacc8d4c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:25.869750 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115374f-232d-4099-9f0c-7f4bacc8d4c3 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:25.940585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1009/2021] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:25 2022] POST /identity/v3/users => generated 294 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:25.942200 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115374f-232d-4099-9f0c-7f4bacc8d4c3 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:25.985896 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115374f-232d-4099-9f0c-7f4bacc8d4c3 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:25.992438 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1013/2022] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:25.993447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd93604a-212e-4c71-a0d8-44e34a722832 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:26.012434 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fd93604a-212e-4c71-a0d8-44e34a722832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:26.050334 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076c960-3e5c-4d4e-b3be-3bc02abeaebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:26.051785 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076c960-3e5c-4d4e-b3be-3bc02abeaebc admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:26.052516 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076c960-3e5c-4d4e-b3be-3bc02abeaebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:26.052968 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076c960-3e5c-4d4e-b3be-3bc02abeaebc admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.054246 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076c960-3e5c-4d4e-b3be-3bc02abeaebc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:26.054747 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076c960-3e5c-4d4e-b3be-3bc02abeaebc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:26.057252 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd93604a-212e-4c71-a0d8-44e34a722832 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:26.062782 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076c960-3e5c-4d4e-b3be-3bc02abeaebc admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:26.069038 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d661e-84f9-458f-8a05-fef24a1555a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:26.072847 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d661e-84f9-458f-8a05-fef24a1555a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:26.072847 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d661e-84f9-458f-8a05-fef24a1555a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:26.072847 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d661e-84f9-458f-8a05-fef24a1555a8 admin admin] PATH_INFO: `/v3/projects/e4047e4eb89d47ddbe4c4091b610b6c6` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.072847 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542d661e-84f9-458f-8a05-fef24a1555a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4047e4eb89d47ddbe4c4091b610b6c6)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:26.077639 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542d661e-84f9-458f-8a05-fef24a1555a8 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:26.105150 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1010/2023] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:09:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:26.105150 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076c960-3e5c-4d4e-b3be-3bc02abeaebc admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:26.118771 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076c960-3e5c-4d4e-b3be-3bc02abeaebc admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:26.129242 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a548c-28e1-4f19-bb31-32ca3e913255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:26.130110 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a548c-28e1-4f19-bb31-32ca3e913255 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:26.130394 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a548c-28e1-4f19-bb31-32ca3e913255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:26.130665 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a548c-28e1-4f19-bb31-32ca3e913255 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.131897 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a548c-28e1-4f19-bb31-32ca3e913255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:26.133229 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a548c-28e1-4f19-bb31-32ca3e913255 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:26.136985 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:26.153995 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1011/2024] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:26.156653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4016b4-4fb7-487d-ad91-c77cabb000c1 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:26.157311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4016b4-4fb7-487d-ad91-c77cabb000c1 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:26.158059 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4016b4-4fb7-487d-ad91-c77cabb000c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.192427 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-542d661e-84f9-458f-8a05-fef24a1555a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4047e4eb89d47ddbe4c4091b610b6c6', 'request_id': 'req-542d661e-84f9-458f-8a05-fef24a1555a8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:26.197412 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-5c4016b4-4fb7-487d-ad91-c77cabb000c1 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:26.200099 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5c4016b4-4fb7-487d-ad91-c77cabb000c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:26.200805 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-542d661e-84f9-458f-8a05-fef24a1555a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4047e4eb89d47ddbe4c4091b610b6c6', 'request_id': 'req-542d661e-84f9-458f-8a05-fef24a1555a8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:26.216050 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-542d661e-84f9-458f-8a05-fef24a1555a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4047e4eb89d47ddbe4c4091b610b6c6', 'request_id': 'req-542d661e-84f9-458f-8a05-fef24a1555a8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:26.246579 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-542d661e-84f9-458f-8a05-fef24a1555a8 admin admin] The token cache is being invalidate because project e4047e4eb89d47ddbe4c4091b610b6c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:26.247258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-542d661e-84f9-458f-8a05-fef24a1555a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:26.249713 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-542d661e-84f9-458f-8a05-fef24a1555a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:26.254321 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1014/2025] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:25 2022] DELETE /identity/v3/projects/e4047e4eb89d47ddbe4c4091b610b6c6 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:26.255839 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d661e-84f9-458f-8a05-fef24a1555a8 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:26.258416 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-542d661e-84f9-458f-8a05-fef24a1555a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:26.324153 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1012/2026] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:26.326547 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4016b4-4fb7-487d-ad91-c77cabb000c1 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:26.328636 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5c4016b4-4fb7-487d-ad91-c77cabb000c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:26.346778 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4016b4-4fb7-487d-ad91-c77cabb000c1 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:26.356311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8bf33-c2c4-4571-bbac-27f94ba8ad2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:26.357382 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8bf33-c2c4-4571-bbac-27f94ba8ad2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:26.357818 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8bf33-c2c4-4571-bbac-27f94ba8ad2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:26.358267 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8bf33-c2c4-4571-bbac-27f94ba8ad2f admin admin] PATH_INFO: `/v3/projects/8ff164eed8be4d859de8f16d230575de/users/172e341cc4c24c5990f18f7df2555cc4/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.379802 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d661e-84f9-458f-8a05-fef24a1555a8 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:26.386227 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd8bf33-c2c4-4571-bbac-27f94ba8ad2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ff164eed8be4d859de8f16d230575de, user_id=172e341cc4c24c5990f18f7df2555cc4, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:26.390065 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277c628-b717-44eb-ae78-fb22d4720bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:26.391150 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277c628-b717-44eb-ae78-fb22d4720bc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:26.391546 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd8bf33-c2c4-4571-bbac-27f94ba8ad2f admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:26.391748 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277c628-b717-44eb-ae78-fb22d4720bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:26.392222 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277c628-b717-44eb-ae78-fb22d4720bc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.393767 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6277c628-b717-44eb-ae78-fb22d4720bc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:26.398247 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6277c628-b717-44eb-ae78-fb22d4720bc8 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:26.426188 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1013/2027] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:26 2022] PUT /identity/v3/projects/8ff164eed8be4d859de8f16d230575de/users/172e341cc4c24c5990f18f7df2555cc4/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:26.430179 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8bf33-c2c4-4571-bbac-27f94ba8ad2f admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:26.431759 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-2fd8bf33-c2c4-4571-bbac-27f94ba8ad2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:26.432984 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:26.473745 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1015/2028] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:26 2022] POST /identity/v3/projects => generated 375 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:26.481266 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277c628-b717-44eb-ae78-fb22d4720bc8 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:26.665437 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277c628-b717-44eb-ae78-fb22d4720bc8 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:26.674806 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de75e0-9cdc-4202-a490-f52bddc7563f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:26.675916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de75e0-9cdc-4202-a490-f52bddc7563f admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:26.676496 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de75e0-9cdc-4202-a490-f52bddc7563f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:26.676978 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de75e0-9cdc-4202-a490-f52bddc7563f admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.678438 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8de75e0-9cdc-4202-a490-f52bddc7563f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:26.680193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8de75e0-9cdc-4202-a490-f52bddc7563f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:26.680381 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8bf33-c2c4-4571-bbac-27f94ba8ad2f admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:26.682490 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8de75e0-9cdc-4202-a490-f52bddc7563f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:26.691347 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc09f2-ee0b-424b-a0c9-a527e5788f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:26.692654 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc09f2-ee0b-424b-a0c9-a527e5788f44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:26.693167 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc09f2-ee0b-424b-a0c9-a527e5788f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:26.693842 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc09f2-ee0b-424b-a0c9-a527e5788f44 admin admin] PATH_INFO: `/v3/users/197932865c1a40679dd3f40ac54fe761` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.696781 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc09f2-ee0b-424b-a0c9-a527e5788f44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=197932865c1a40679dd3f40ac54fe761)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:26.701153 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc09f2-ee0b-424b-a0c9-a527e5788f44 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:26.730020 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1016/2029] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:09:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:26.730020 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de75e0-9cdc-4202-a490-f52bddc7563f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:26.795019 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de75e0-9cdc-4202-a490-f52bddc7563f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:26.820916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e02b3-47b2-438c-b78f-62b1c659a6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:26.822432 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e02b3-47b2-438c-b78f-62b1c659a6f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:26.822724 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e02b3-47b2-438c-b78f-62b1c659a6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:26.823000 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e02b3-47b2-438c-b78f-62b1c659a6f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.823798 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e02b3-47b2-438c-b78f-62b1c659a6f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:26.825733 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e02b3-47b2-438c-b78f-62b1c659a6f0 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:26.857548 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1017/2030] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:26.859637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e02b3-47b2-438c-b78f-62b1c659a6f0 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:26.873342 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d2cc09f2-ee0b-424b-a0c9-a527e5788f44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '197932865c1a40679dd3f40ac54fe761'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:26.874534 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d2cc09f2-ee0b-424b-a0c9-a527e5788f44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '197932865c1a40679dd3f40ac54fe761'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:26.888181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e02b3-47b2-438c-b78f-62b1c659a6f0 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:26.892257 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d2cc09f2-ee0b-424b-a0c9-a527e5788f44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '197932865c1a40679dd3f40ac54fe761'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:26.902979 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564745f-e678-4f48-a2eb-93e091502d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:26.904875 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564745f-e678-4f48-a2eb-93e091502d67 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:26.905336 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564745f-e678-4f48-a2eb-93e091502d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:26.906101 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564745f-e678-4f48-a2eb-93e091502d67 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.906485 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d2cc09f2-ee0b-424b-a0c9-a527e5788f44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '197932865c1a40679dd3f40ac54fe761'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:26.908554 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6564745f-e678-4f48-a2eb-93e091502d67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:26.913989 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6564745f-e678-4f48-a2eb-93e091502d67 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:26.925062 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:26.929146 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d2cc09f2-ee0b-424b-a0c9-a527e5788f44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '197932865c1a40679dd3f40ac54fe761'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:26.960017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1014/2031] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:26 2022] DELETE /identity/v3/users/197932865c1a40679dd3f40ac54fe761 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:26.961239 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc09f2-ee0b-424b-a0c9-a527e5788f44 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:26.963166 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d2cc09f2-ee0b-424b-a0c9-a527e5788f44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:26.963842 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1018/2032] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:26 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:26.965481 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564745f-e678-4f48-a2eb-93e091502d67 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:26.970887 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6564745f-e678-4f48-a2eb-93e091502d67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:27.012707 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564745f-e678-4f48-a2eb-93e091502d67 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:27.149518 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc09f2-ee0b-424b-a0c9-a527e5788f44 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:27.167174 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3168ead0-f9f1-42b5-92d2-a1219fc8eb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:27.167845 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168ead0-f9f1-42b5-92d2-a1219fc8eb21 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.168406 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168ead0-f9f1-42b5-92d2-a1219fc8eb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.168584 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168ead0-f9f1-42b5-92d2-a1219fc8eb21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.169404 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3168ead0-f9f1-42b5-92d2-a1219fc8eb21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:27.172156 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3168ead0-f9f1-42b5-92d2-a1219fc8eb21 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:27.259831 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b7365-5665-4516-9105-3d5edfac67c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:27.261553 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b7365-5665-4516-9105-3d5edfac67c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.262116 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b7365-5665-4516-9105-3d5edfac67c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.262701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b7365-5665-4516-9105-3d5edfac67c6 admin admin] PATH_INFO: `/v3/projects/8ff164eed8be4d859de8f16d230575de/users/172e341cc4c24c5990f18f7df2555cc4/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.263909 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1015/2033] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:26 2022] POST /identity/v3/projects => generated 385 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:27.265257 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3168ead0-f9f1-42b5-92d2-a1219fc8eb21 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:27.267263 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3168ead0-f9f1-42b5-92d2-a1219fc8eb21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:27.274834 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b7365-5665-4516-9105-3d5edfac67c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ff164eed8be4d859de8f16d230575de, user_id=172e341cc4c24c5990f18f7df2555cc4, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:27.286274 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b7365-5665-4516-9105-3d5edfac67c6 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:27.324203 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1019/2034] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:26 2022] PUT /identity/v3/projects/8ff164eed8be4d859de8f16d230575de/users/172e341cc4c24c5990f18f7df2555cc4/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:27.325231 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b7365-5665-4516-9105-3d5edfac67c6 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:27.424854 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:27.434543 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3168ead0-f9f1-42b5-92d2-a1219fc8eb21 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:27.472748 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b7365-5665-4516-9105-3d5edfac67c6 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:27.477818 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc292f0-e0f6-43c7-bd99-78628f2424e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:27.478490 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc292f0-e0f6-43c7-bd99-78628f2424e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.479113 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc292f0-e0f6-43c7-bd99-78628f2424e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.479550 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc292f0-e0f6-43c7-bd99-78628f2424e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.480439 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc292f0-e0f6-43c7-bd99-78628f2424e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:27.483819 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc292f0-e0f6-43c7-bd99-78628f2424e7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:27.506851 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724999e-b238-4956-b759-0a4215688398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:27.507644 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724999e-b238-4956-b759-0a4215688398 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.507927 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724999e-b238-4956-b759-0a4215688398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.508204 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724999e-b238-4956-b759-0a4215688398 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.509047 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724999e-b238-4956-b759-0a4215688398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:27.511165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724999e-b238-4956-b759-0a4215688398 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:27.527203 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1016/2035] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:27.528967 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e201175-58f0-443c-9332-4c2d7626aac9 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.529265 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e201175-58f0-443c-9332-4c2d7626aac9 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.529547 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e201175-58f0-443c-9332-4c2d7626aac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.563516 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1020/2036] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:27 2022] POST /identity/v3/users => generated 293 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:27.567130 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc292f0-e0f6-43c7-bd99-78628f2424e7 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:27.587661 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-4e201175-58f0-443c-9332-4c2d7626aac9 None None] MFA Rules not processed for user `172e341cc4c24c5990f18f7df2555cc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:27.591272 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc292f0-e0f6-43c7-bd99-78628f2424e7 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:27.591939 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-4e201175-58f0-443c-9332-4c2d7626aac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:27.607169 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860db15-6197-45b1-9e6d-5f6b3bf95cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:27.607986 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860db15-6197-45b1-9e6d-5f6b3bf95cae admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.608433 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860db15-6197-45b1-9e6d-5f6b3bf95cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.608952 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860db15-6197-45b1-9e6d-5f6b3bf95cae admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.610521 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6860db15-6197-45b1-9e6d-5f6b3bf95cae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:27.613686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6860db15-6197-45b1-9e6d-5f6b3bf95cae admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:27.616857 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:27.635808 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1021/2037] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:27.637404 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860db15-6197-45b1-9e6d-5f6b3bf95cae admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:27.656152 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860db15-6197-45b1-9e6d-5f6b3bf95cae admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:27.667831 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5710a283-990b-4b85-92e5-6abaefbe9a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:27.669149 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5710a283-990b-4b85-92e5-6abaefbe9a47 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.669662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5710a283-990b-4b85-92e5-6abaefbe9a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.670184 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5710a283-990b-4b85-92e5-6abaefbe9a47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.671847 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5710a283-990b-4b85-92e5-6abaefbe9a47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:27.675301 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5710a283-990b-4b85-92e5-6abaefbe9a47 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:27.694683 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1022/2038] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:27.700863 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5710a283-990b-4b85-92e5-6abaefbe9a47 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:27.725135 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5710a283-990b-4b85-92e5-6abaefbe9a47 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:27.739550 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9fd8cc-8aac-4e78-aa8f-7141c6cb575e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:27.740933 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9fd8cc-8aac-4e78-aa8f-7141c6cb575e admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.741523 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9fd8cc-8aac-4e78-aa8f-7141c6cb575e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.742150 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9fd8cc-8aac-4e78-aa8f-7141c6cb575e admin admin] PATH_INFO: `/v3/projects/cf0e090327da468196d349b6213848e9/users/55f61f8ca6d547569a1457baafe8aa27/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.744532 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1017/2039] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:27 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:27.745427 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e201175-58f0-443c-9332-4c2d7626aac9 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:27.781146 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e201175-58f0-443c-9332-4c2d7626aac9 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:27.782517 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9fd8cc-8aac-4e78-aa8f-7141c6cb575e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf0e090327da468196d349b6213848e9, user_id=55f61f8ca6d547569a1457baafe8aa27, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:27.786156 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9232bc91-e980-4e3d-996f-62c387940f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:27.786641 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9fd8cc-8aac-4e78-aa8f-7141c6cb575e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:27.786811 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232bc91-e980-4e3d-996f-62c387940f92 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.787096 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232bc91-e980-4e3d-996f-62c387940f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.787387 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232bc91-e980-4e3d-996f-62c387940f92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.788256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9232bc91-e980-4e3d-996f-62c387940f92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:27.790089 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9232bc91-e980-4e3d-996f-62c387940f92 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:27.804238 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1018/2040] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:27.805349 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9232bc91-e980-4e3d-996f-62c387940f92 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:27.807863 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-9232bc91-e980-4e3d-996f-62c387940f92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:27.814219 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1023/2041] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:27 2022] PUT /identity/v3/projects/cf0e090327da468196d349b6213848e9/users/55f61f8ca6d547569a1457baafe8aa27/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:27.815580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9fd8cc-8aac-4e78-aa8f-7141c6cb575e admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:27.824308 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9232bc91-e980-4e3d-996f-62c387940f92 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:27.826547 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8c9fd8cc-8aac-4e78-aa8f-7141c6cb575e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:27.915635 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9fd8cc-8aac-4e78-aa8f-7141c6cb575e admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:27.920519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12735256-6208-43b7-b685-705fd0be2ad5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:27.921346 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12735256-6208-43b7-b685-705fd0be2ad5 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.921670 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12735256-6208-43b7-b685-705fd0be2ad5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.922022 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12735256-6208-43b7-b685-705fd0be2ad5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.924082 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-12735256-6208-43b7-b685-705fd0be2ad5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:27.930781 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b00b42-96a2-477d-b2b4-03dbc41735ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:27.931553 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b00b42-96a2-477d-b2b4-03dbc41735ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.931985 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b00b42-96a2-477d-b2b4-03dbc41735ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.932251 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b00b42-96a2-477d-b2b4-03dbc41735ac admin admin] PATH_INFO: `/v3/projects/72623015993c45e9b1f31fa64dae8435` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.933702 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b00b42-96a2-477d-b2b4-03dbc41735ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=72623015993c45e9b1f31fa64dae8435)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:27.935847 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b00b42-96a2-477d-b2b4-03dbc41735ac admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:28.010278 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-99b00b42-96a2-477d-b2b4-03dbc41735ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72623015993c45e9b1f31fa64dae8435', 'request_id': 'req-99b00b42-96a2-477d-b2b4-03dbc41735ac'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:28.012778 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12735256-6208-43b7-b685-705fd0be2ad5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:28.015919 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12735256-6208-43b7-b685-705fd0be2ad5 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:28.021928 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-99b00b42-96a2-477d-b2b4-03dbc41735ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72623015993c45e9b1f31fa64dae8435', 'request_id': 'req-99b00b42-96a2-477d-b2b4-03dbc41735ac'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:28.027088 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1024/2042] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:28.028296 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12735256-6208-43b7-b685-705fd0be2ad5 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:28.035529 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-99b00b42-96a2-477d-b2b4-03dbc41735ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72623015993c45e9b1f31fa64dae8435', 'request_id': 'req-99b00b42-96a2-477d-b2b4-03dbc41735ac'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:28.041908 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12735256-6208-43b7-b685-705fd0be2ad5 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:28.061265 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-99b00b42-96a2-477d-b2b4-03dbc41735ac admin admin] The token cache is being invalidate because project 72623015993c45e9b1f31fa64dae8435 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:28.061588 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-99b00b42-96a2-477d-b2b4-03dbc41735ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:28.062340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-99b00b42-96a2-477d-b2b4-03dbc41735ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:28.065546 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1019/2043] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:27 2022] DELETE /identity/v3/projects/72623015993c45e9b1f31fa64dae8435 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:28.066642 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0cc5e7-2b7a-46c9-80f3-84cef95721dc None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:28.066924 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0cc5e7-2b7a-46c9-80f3-84cef95721dc None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:28.067201 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0cc5e7-2b7a-46c9-80f3-84cef95721dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:28.104919 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-7a0cc5e7-2b7a-46c9-80f3-84cef95721dc None None] MFA Rules not processed for user `55f61f8ca6d547569a1457baafe8aa27`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:28.106686 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7a0cc5e7-2b7a-46c9-80f3-84cef95721dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:28.112527 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:28.129146 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:28.139694 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b486b687-c24b-4d3c-a125-cdb2fa70dfa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:28.140471 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486b687-c24b-4d3c-a125-cdb2fa70dfa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:28.140754 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486b687-c24b-4d3c-a125-cdb2fa70dfa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:28.141034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486b687-c24b-4d3c-a125-cdb2fa70dfa5 admin admin] PATH_INFO: `/v3/projects/f812076f1abf4f03a2708691c93560b1/users/240ab5affd3c4ae8a85ba4148428e607/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:28.167236 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b486b687-c24b-4d3c-a125-cdb2fa70dfa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f812076f1abf4f03a2708691c93560b1, user_id=240ab5affd3c4ae8a85ba4148428e607, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:28.170974 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b486b687-c24b-4d3c-a125-cdb2fa70dfa5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:28.191905 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1025/2044] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:28 2022] PUT /identity/v3/projects/f812076f1abf4f03a2708691c93560b1/users/240ab5affd3c4ae8a85ba4148428e607/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:28.194674 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d738435-8422-4184-920c-e398f0d5e9ea None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:28.195018 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d738435-8422-4184-920c-e398f0d5e9ea None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:28.195352 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d738435-8422-4184-920c-e398f0d5e9ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:28.214250 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1020/2045] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:28.216134 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dbfb58-c6c7-4497-a707-7a7cf937c254 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:28.216658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dbfb58-c6c7-4497-a707-7a7cf937c254 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:28.216933 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dbfb58-c6c7-4497-a707-7a7cf937c254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:28.241221 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-1d738435-8422-4184-920c-e398f0d5e9ea None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:28.243761 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1d738435-8422-4184-920c-e398f0d5e9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:28.263924 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-b3dbfb58-c6c7-4497-a707-7a7cf937c254 None None] MFA Rules not processed for user `240ab5affd3c4ae8a85ba4148428e607`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:28.266444 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b3dbfb58-c6c7-4497-a707-7a7cf937c254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:28.390520 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1026/2046] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:28.390520 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d738435-8422-4184-920c-e398f0d5e9ea None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:28.390520 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1d738435-8422-4184-920c-e398f0d5e9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:28.404556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1021/2047] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:28.405979 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfb42e-c1f5-476c-a0b1-3372133ce606 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:28.406619 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfb42e-c1f5-476c-a0b1-3372133ce606 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:28.406880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfb42e-c1f5-476c-a0b1-3372133ce606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:28.444242 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-cccfb42e-c1f5-476c-a0b1-3372133ce606 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:28.446116 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-cccfb42e-c1f5-476c-a0b1-3372133ce606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:28.460133 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1022/2048] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:28.461972 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccfb42e-c1f5-476c-a0b1-3372133ce606 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:28.464743 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-cccfb42e-c1f5-476c-a0b1-3372133ce606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:28.484361 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccfb42e-c1f5-476c-a0b1-3372133ce606 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:28.493058 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cca1d-1637-40c5-8540-095679806324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:28.494449 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cca1d-1637-40c5-8540-095679806324 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:28.494991 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cca1d-1637-40c5-8540-095679806324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:28.495399 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cca1d-1637-40c5-8540-095679806324 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:28.496620 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45cca1d-1637-40c5-8540-095679806324 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:28.507370 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45cca1d-1637-40c5-8540-095679806324 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:28.576681 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1023/2049] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:28 2022] POST /identity/v3/projects => generated 387 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:28.578424 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cca1d-1637-40c5-8540-095679806324 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:28.604500 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d738435-8422-4184-920c-e398f0d5e9ea None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:28.704674 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b4f75-7501-4b1d-b185-d8eb3839e1d8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7317c8829cb4fa1b30711f1db575cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:28.705320 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b4f75-7501-4b1d-b185-d8eb3839e1d8 service glance] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:28.705594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b4f75-7501-4b1d-b185-d8eb3839e1d8 service glance] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:28.705901 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b4f75-7501-4b1d-b185-d8eb3839e1d8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:28.732729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:28.747896 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cca1d-1637-40c5-8540-095679806324 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:28.753161 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db618086-bbf4-46a8-97d8-0cac6172d73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:28.754135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db618086-bbf4-46a8-97d8-0cac6172d73f admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:28.754424 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db618086-bbf4-46a8-97d8-0cac6172d73f admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:28.754700 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db618086-bbf4-46a8-97d8-0cac6172d73f admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:28.755436 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db618086-bbf4-46a8-97d8-0cac6172d73f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:28.755829 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db618086-bbf4-46a8-97d8-0cac6172d73f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:28.757905 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db618086-bbf4-46a8-97d8-0cac6172d73f admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:28.776019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1024/2050] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:09:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:28.777172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db618086-bbf4-46a8-97d8-0cac6172d73f admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:28.789910 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db618086-bbf4-46a8-97d8-0cac6172d73f admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:28.798618 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b4f75-7501-4b1d-b185-d8eb3839e1d8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:28.799077 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ffc2b-26ed-4e25-81d2-e35ea531daef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:28.799835 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ffc2b-26ed-4e25-81d2-e35ea531daef admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:28.800076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ffc2b-26ed-4e25-81d2-e35ea531daef admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:28.800542 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ffc2b-26ed-4e25-81d2-e35ea531daef admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:28.801320 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ffc2b-26ed-4e25-81d2-e35ea531daef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:28.801890 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b4f75-7501-4b1d-b185-d8eb3839e1d8 service glance] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:28.803551 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ffc2b-26ed-4e25-81d2-e35ea531daef admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:28.816362 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:28.819302 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1027/2051] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:28.820736 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b4f75-7501-4b1d-b185-d8eb3839e1d8 service glance] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:28.844907 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b4f75-7501-4b1d-b185-d8eb3839e1d8 service glance] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:28.846637 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1025/2052] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:28 2022] POST /identity/v3/users => generated 294 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:28.851271 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ffc2b-26ed-4e25-81d2-e35ea531daef admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:28.854047 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904fd51-ad88-431d-a05c-6bf4c5bb7884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:28.855629 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904fd51-ad88-431d-a05c-6bf4c5bb7884 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:28.856162 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904fd51-ad88-431d-a05c-6bf4c5bb7884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:28.857062 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904fd51-ad88-431d-a05c-6bf4c5bb7884 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:28.859177 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8904fd51-ad88-431d-a05c-6bf4c5bb7884 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:28.876976 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ffc2b-26ed-4e25-81d2-e35ea531daef admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:28.878636 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8904fd51-ad88-431d-a05c-6bf4c5bb7884 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:28.888435 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa9ca1-c383-4a74-9311-3fe8ea31273d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:28.889813 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa9ca1-c383-4a74-9311-3fe8ea31273d admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:28.890590 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa9ca1-c383-4a74-9311-3fe8ea31273d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:28.890850 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa9ca1-c383-4a74-9311-3fe8ea31273d admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:28.892116 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa9ca1-c383-4a74-9311-3fe8ea31273d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:28.898212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa9ca1-c383-4a74-9311-3fe8ea31273d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:28.928474 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1026/2053] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:28.931959 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa9ca1-c383-4a74-9311-3fe8ea31273d admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:28.947345 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa9ca1-c383-4a74-9311-3fe8ea31273d admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:28.955696 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f92ce7-3cfa-440c-b724-6ea4284223f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:28.957010 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f92ce7-3cfa-440c-b724-6ea4284223f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:28.957343 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f92ce7-3cfa-440c-b724-6ea4284223f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:28.957887 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f92ce7-3cfa-440c-b724-6ea4284223f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:28.960326 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f92ce7-3cfa-440c-b724-6ea4284223f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:28.962749 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f92ce7-3cfa-440c-b724-6ea4284223f8 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:28.972347 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1028/2054] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:28 2022] POST /identity/v3/projects => generated 371 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:28.974966 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904fd51-ad88-431d-a05c-6bf4c5bb7884 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:28.983374 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1027/2055] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:28.990908 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f92ce7-3cfa-440c-b724-6ea4284223f8 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:29.012326 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f92ce7-3cfa-440c-b724-6ea4284223f8 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:29.092325 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904fd51-ad88-431d-a05c-6bf4c5bb7884 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:29.099215 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3671b67-7c5f-4d01-8fa0-05eb46fbf618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:29.099215 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3671b67-7c5f-4d01-8fa0-05eb46fbf618 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:29.099215 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3671b67-7c5f-4d01-8fa0-05eb46fbf618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:29.099719 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3671b67-7c5f-4d01-8fa0-05eb46fbf618 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:29.100209 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3671b67-7c5f-4d01-8fa0-05eb46fbf618 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:29.102265 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3671b67-7c5f-4d01-8fa0-05eb46fbf618 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:29.131452 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1029/2056] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:28 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:29.135954 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3671b67-7c5f-4d01-8fa0-05eb46fbf618 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:29.150109 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3671b67-7c5f-4d01-8fa0-05eb46fbf618 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:29.157456 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee9d53a-ae4f-412b-8124-cd12674ac642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:29.158168 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9d53a-ae4f-412b-8124-cd12674ac642 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:29.158469 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9d53a-ae4f-412b-8124-cd12674ac642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:29.158772 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9d53a-ae4f-412b-8124-cd12674ac642 admin admin] PATH_INFO: `/v3/projects/6636e1a220944e0ea518b444ffc5b784/users/b3875fbc3cbc4ee19a89a79cdd6c2753/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:29.162254 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db9c01-0829-4075-a9d1-448e66059755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:29.162906 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db9c01-0829-4075-a9d1-448e66059755 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:29.163184 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db9c01-0829-4075-a9d1-448e66059755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:29.163465 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db9c01-0829-4075-a9d1-448e66059755 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:29.164165 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db9c01-0829-4075-a9d1-448e66059755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:29.166198 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db9c01-0829-4075-a9d1-448e66059755 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:29.182415 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1030/2057] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:29.187876 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db9c01-0829-4075-a9d1-448e66059755 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:29.195492 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee9d53a-ae4f-412b-8124-cd12674ac642 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6636e1a220944e0ea518b444ffc5b784, user_id=b3875fbc3cbc4ee19a89a79cdd6c2753, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:29.201395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee9d53a-ae4f-412b-8124-cd12674ac642 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:29.202621 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db9c01-0829-4075-a9d1-448e66059755 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:29.213596 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba381517-d8cf-4522-b097-b9daba380000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:29.214363 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba381517-d8cf-4522-b097-b9daba380000 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:29.214988 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba381517-d8cf-4522-b097-b9daba380000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:29.215372 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba381517-d8cf-4522-b097-b9daba380000 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:29.216160 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba381517-d8cf-4522-b097-b9daba380000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:29.219277 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba381517-d8cf-4522-b097-b9daba380000 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:29.228390 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1028/2058] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:28 2022] PUT /identity/v3/projects/6636e1a220944e0ea518b444ffc5b784/users/b3875fbc3cbc4ee19a89a79cdd6c2753/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:29.232991 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1031/2059] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:29.237607 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba381517-d8cf-4522-b097-b9daba380000 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:29.238537 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d1e52-60b0-4e62-b639-8f650b98ba97 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:29.239069 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d1e52-60b0-4e62-b639-8f650b98ba97 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:29.239347 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d1e52-60b0-4e62-b639-8f650b98ba97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:29.252845 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba381517-d8cf-4522-b097-b9daba380000 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:29.280257 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-8e9d1e52-60b0-4e62-b639-8f650b98ba97 None None] MFA Rules not processed for user `b3875fbc3cbc4ee19a89a79cdd6c2753`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:29.282610 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-8e9d1e52-60b0-4e62-b639-8f650b98ba97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:29.350665 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d56906-8294-4a6c-8e3d-6e4604b4e2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:29.351409 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d56906-8294-4a6c-8e3d-6e4604b4e2fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:29.351734 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d56906-8294-4a6c-8e3d-6e4604b4e2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:29.351989 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d56906-8294-4a6c-8e3d-6e4604b4e2fb admin admin] PATH_INFO: `/v3/projects/5166861cf436411e9797c575c32bb800/users/e317f73780184baa87733294b8db5fe7/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:29.366717 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:29.373254 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d56906-8294-4a6c-8e3d-6e4604b4e2fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5166861cf436411e9797c575c32bb800, user_id=e317f73780184baa87733294b8db5fe7, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:29.376208 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d56906-8294-4a6c-8e3d-6e4604b4e2fb admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:29.386049 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1029/2060] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:29.391238 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d1e52-60b0-4e62-b639-8f650b98ba97 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:29.399832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1032/2061] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:29 2022] PUT /identity/v3/projects/5166861cf436411e9797c575c32bb800/users/e317f73780184baa87733294b8db5fe7/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:29.405731 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af9951-6b14-4c47-a74a-3e6f571e0217 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:29.406059 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af9951-6b14-4c47-a74a-3e6f571e0217 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:29.406528 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af9951-6b14-4c47-a74a-3e6f571e0217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:29.408774 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d1e52-60b0-4e62-b639-8f650b98ba97 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:29.429735 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-68af9951-6b14-4c47-a74a-3e6f571e0217 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:29.431166 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-68af9951-6b14-4c47-a74a-3e6f571e0217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:29.507079 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:29.525601 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1033/2062] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:29.536670 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af9951-6b14-4c47-a74a-3e6f571e0217 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:29.537442 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85779c8-1265-4e09-85b1-a1715776e321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:29.538361 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85779c8-1265-4e09-85b1-a1715776e321 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:29.538804 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85779c8-1265-4e09-85b1-a1715776e321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:29.539082 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85779c8-1265-4e09-85b1-a1715776e321 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:29.539863 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85779c8-1265-4e09-85b1-a1715776e321 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:29.542075 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85779c8-1265-4e09-85b1-a1715776e321 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:29.548130 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-68af9951-6b14-4c47-a74a-3e6f571e0217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:29.597561 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1030/2063] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:29 2022] POST /identity/v3/projects => generated 387 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:09:29.605511 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85779c8-1265-4e09-85b1-a1715776e321 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:29.677774 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af9951-6b14-4c47-a74a-3e6f571e0217 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:29.728029 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85779c8-1265-4e09-85b1-a1715776e321 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:29.735319 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247df5b-524c-4fb6-ae0e-559925c0d5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:29.736377 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247df5b-524c-4fb6-ae0e-559925c0d5d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:29.736824 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247df5b-524c-4fb6-ae0e-559925c0d5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:29.737268 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247df5b-524c-4fb6-ae0e-559925c0d5d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:29.738860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9247df5b-524c-4fb6-ae0e-559925c0d5d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:29.744886 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9247df5b-524c-4fb6-ae0e-559925c0d5d4 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:29.766447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813bdcc1-acd0-4527-81d0-452706ebe611 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:29.767644 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813bdcc1-acd0-4527-81d0-452706ebe611 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:29.767980 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813bdcc1-acd0-4527-81d0-452706ebe611 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:29.768552 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813bdcc1-acd0-4527-81d0-452706ebe611 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:29.781931 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813bdcc1-acd0-4527-81d0-452706ebe611 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:29.783593 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813bdcc1-acd0-4527-81d0-452706ebe611 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:29.792102 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1031/2064] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:29 2022] POST /identity/v3/users => generated 300 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:29.796842 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247df5b-524c-4fb6-ae0e-559925c0d5d4 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:29.800180 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1034/2065] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:09:29.819780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247df5b-524c-4fb6-ae0e-559925c0d5d4 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:29.829037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0596455-ee9f-4d61-89cb-ed0fc11052f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:29.831841 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0596455-ee9f-4d61-89cb-ed0fc11052f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:29.832906 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0596455-ee9f-4d61-89cb-ed0fc11052f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:29.836990 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0596455-ee9f-4d61-89cb-ed0fc11052f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:29.836990 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0596455-ee9f-4d61-89cb-ed0fc11052f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:29.839180 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0596455-ee9f-4d61-89cb-ed0fc11052f7 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:29.875571 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813bdcc1-acd0-4527-81d0-452706ebe611 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:29.884074 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1032/2066] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:29.889533 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813bdcc1-acd0-4527-81d0-452706ebe611 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:29.901792 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce9347-89bb-4d92-a426-ee0e5a81b5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:29.917181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce9347-89bb-4d92-a426-ee0e5a81b5b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:29.917705 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce9347-89bb-4d92-a426-ee0e5a81b5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:29.918010 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce9347-89bb-4d92-a426-ee0e5a81b5b9 admin admin] PATH_INFO: `/v3/projects/32eeebdbc7d64f319568c7808cf620d4/users/fff9c5fa618e4f65b24b052704041221/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:29.960047 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ce9347-89bb-4d92-a426-ee0e5a81b5b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32eeebdbc7d64f319568c7808cf620d4, user_id=fff9c5fa618e4f65b24b052704041221, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:29.965269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0596455-ee9f-4d61-89cb-ed0fc11052f7 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:29.967461 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b0596455-ee9f-4d61-89cb-ed0fc11052f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:29.978529 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ce9347-89bb-4d92-a426-ee0e5a81b5b9 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:29.997921 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1035/2067] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:29 2022] PUT /identity/v3/projects/32eeebdbc7d64f319568c7808cf620d4/users/fff9c5fa618e4f65b24b052704041221/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:30.002515 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce9347-89bb-4d92-a426-ee0e5a81b5b9 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:30.048179 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:30.115194 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce9347-89bb-4d92-a426-ee0e5a81b5b9 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:30.124858 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0596455-ee9f-4d61-89cb-ed0fc11052f7 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:30.126981 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b935d4c3-074a-42db-aef8-37464c21139f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:30.128387 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935d4c3-074a-42db-aef8-37464c21139f admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:30.129063 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935d4c3-074a-42db-aef8-37464c21139f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:30.129310 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935d4c3-074a-42db-aef8-37464c21139f admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:30.130607 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b935d4c3-074a-42db-aef8-37464c21139f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:30.133312 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b935d4c3-074a-42db-aef8-37464c21139f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:30.153945 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1036/2068] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:30 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:30.155190 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b935d4c3-074a-42db-aef8-37464c21139f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:30.188486 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b935d4c3-074a-42db-aef8-37464c21139f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:30.197409 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813fa177-abe3-48b0-ace7-9d81b0dcac8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:30.198928 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813fa177-abe3-48b0-ace7-9d81b0dcac8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:30.199797 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813fa177-abe3-48b0-ace7-9d81b0dcac8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:30.200091 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813fa177-abe3-48b0-ace7-9d81b0dcac8d admin admin] PATH_INFO: `/v3/projects/32eeebdbc7d64f319568c7808cf620d4/users/fff9c5fa618e4f65b24b052704041221/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:30.204207 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813fa177-abe3-48b0-ace7-9d81b0dcac8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=32eeebdbc7d64f319568c7808cf620d4, user_id=fff9c5fa618e4f65b24b052704041221, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:30.208168 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813fa177-abe3-48b0-ace7-9d81b0dcac8d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:30.217259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb63a47-5fa0-45cb-9b19-d95498b464d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:30.217259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb63a47-5fa0-45cb-9b19-d95498b464d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:30.217259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb63a47-5fa0-45cb-9b19-d95498b464d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:30.217259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb63a47-5fa0-45cb-9b19-d95498b464d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:30.219110 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-fcb63a47-5fa0-45cb-9b19-d95498b464d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:30.235023 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1037/2069] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:30 2022] PUT /identity/v3/projects/32eeebdbc7d64f319568c7808cf620d4/users/fff9c5fa618e4f65b24b052704041221/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:30.252378 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769cc3f-c79e-432d-a00a-97015b3b2316 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:30.252378 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769cc3f-c79e-432d-a00a-97015b3b2316 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:30.252378 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769cc3f-c79e-432d-a00a-97015b3b2316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:30.257512 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:30.296869 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8769cc3f-c79e-432d-a00a-97015b3b2316 None None] MFA Rules not processed for user `fff9c5fa618e4f65b24b052704041221`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:30.300674 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8769cc3f-c79e-432d-a00a-97015b3b2316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:30.335212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb63a47-5fa0-45cb-9b19-d95498b464d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:30.337572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb63a47-5fa0-45cb-9b19-d95498b464d4 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:30.350043 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1033/2070] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:30.451908 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1038/2071] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:30 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:30.461379 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb63a47-5fa0-45cb-9b19-d95498b464d4 service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:30.463468 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-fcb63a47-5fa0-45cb-9b19-d95498b464d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:30.483313 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb63a47-5fa0-45cb-9b19-d95498b464d4 service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:30.607193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180398f8-4296-4354-a02f-65dbe697e6dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:30.607919 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180398f8-4296-4354-a02f-65dbe697e6dd service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:30.608178 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180398f8-4296-4354-a02f-65dbe697e6dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:30.608478 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180398f8-4296-4354-a02f-65dbe697e6dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:30.628123 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180398f8-4296-4354-a02f-65dbe697e6dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:30.635621 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180398f8-4296-4354-a02f-65dbe697e6dd service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:30.657466 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1034/2072] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:30 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:31.176716 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd26e0d-d837-4a7a-9df1-980b6b9abc75 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:31.177496 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd26e0d-d837-4a7a-9df1-980b6b9abc75 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:31.177496 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd26e0d-d837-4a7a-9df1-980b6b9abc75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:31.278051 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-dcd26e0d-d837-4a7a-9df1-980b6b9abc75 None None] MFA Rules not processed for user `e317f73780184baa87733294b8db5fe7`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:31.278051 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-dcd26e0d-d837-4a7a-9df1-980b6b9abc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:31.495327 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1039/2073] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:31.501623 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180398f8-4296-4354-a02f-65dbe697e6dd service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:31.596068 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd26e0d-d837-4a7a-9df1-980b6b9abc75 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:31.623693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd26e0d-d837-4a7a-9df1-980b6b9abc75 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:31.629231 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c3673-307f-4568-87da-8e0393068604 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:31.630078 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c3673-307f-4568-87da-8e0393068604 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:31.630332 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c3673-307f-4568-87da-8e0393068604 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:31.630627 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c3673-307f-4568-87da-8e0393068604 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:31.636733 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c94c3673-307f-4568-87da-8e0393068604 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:31.686615 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:31.752136 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180398f8-4296-4354-a02f-65dbe697e6dd service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:31.772809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c327d6a-3b90-415e-97b8-c57a92d4cbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:31.777558 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c327d6a-3b90-415e-97b8-c57a92d4cbf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:31.778406 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c327d6a-3b90-415e-97b8-c57a92d4cbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:31.779987 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c327d6a-3b90-415e-97b8-c57a92d4cbf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:31.784746 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c327d6a-3b90-415e-97b8-c57a92d4cbf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:31.788792 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c327d6a-3b90-415e-97b8-c57a92d4cbf5 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:31.814459 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c3673-307f-4568-87da-8e0393068604 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:31.816219 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c3673-307f-4568-87da-8e0393068604 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:31.825336 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1035/2074] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:31 2022] POST /identity/v3/projects => generated 371 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:09:31.826722 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c327d6a-3b90-415e-97b8-c57a92d4cbf5 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:31.828630 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1040/2075] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:31 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:09:31.831088 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c3673-307f-4568-87da-8e0393068604 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:31.845362 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c3673-307f-4568-87da-8e0393068604 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:31.946296 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:31.948777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4bbca7-b48a-4357-9315-00c0daf6ba11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:31.949794 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4bbca7-b48a-4357-9315-00c0daf6ba11 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:31.950223 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4bbca7-b48a-4357-9315-00c0daf6ba11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:31.950693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4bbca7-b48a-4357-9315-00c0daf6ba11 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:31.951967 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4bbca7-b48a-4357-9315-00c0daf6ba11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:31.956084 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4bbca7-b48a-4357-9315-00c0daf6ba11 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:31.968881 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c327d6a-3b90-415e-97b8-c57a92d4cbf5 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:31.976116 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52217f2-8815-4f2d-a3d1-1a2ce3761e0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:31.977177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52217f2-8815-4f2d-a3d1-1a2ce3761e0f service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:31.977558 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52217f2-8815-4f2d-a3d1-1a2ce3761e0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:31.978050 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52217f2-8815-4f2d-a3d1-1a2ce3761e0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:32.021199 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1041/2076] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:31 2022] POST /identity/v3/users => generated 292 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:09:32.022767 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4bbca7-b48a-4357-9315-00c0daf6ba11 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:32.046180 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4bbca7-b48a-4357-9315-00c0daf6ba11 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:32.055774 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca366dde-51d8-456f-8706-7470fad047b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:32.056249 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca366dde-51d8-456f-8706-7470fad047b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:32.056699 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca366dde-51d8-456f-8706-7470fad047b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:32.057029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca366dde-51d8-456f-8706-7470fad047b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:32.057762 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca366dde-51d8-456f-8706-7470fad047b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:32.059429 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca366dde-51d8-456f-8706-7470fad047b7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:32.074406 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1042/2077] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:32 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:32.079428 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca366dde-51d8-456f-8706-7470fad047b7 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:32.102342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca366dde-51d8-456f-8706-7470fad047b7 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:32.107901 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17731acf-f169-4006-a088-36eb3e5ac513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:32.108769 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17731acf-f169-4006-a088-36eb3e5ac513 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:32.109239 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17731acf-f169-4006-a088-36eb3e5ac513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:32.109516 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17731acf-f169-4006-a088-36eb3e5ac513 admin admin] PATH_INFO: `/v3/projects/a687a9ada4494ab4b93aa989fe7b47a2/users/c44dad19055447d9a24206c441e2a0ab/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:32.117995 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52217f2-8815-4f2d-a3d1-1a2ce3761e0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:32.122145 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52217f2-8815-4f2d-a3d1-1a2ce3761e0f service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:32.131762 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17731acf-f169-4006-a088-36eb3e5ac513 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a687a9ada4494ab4b93aa989fe7b47a2, user_id=c44dad19055447d9a24206c441e2a0ab, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:32.134586 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17731acf-f169-4006-a088-36eb3e5ac513 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:32.136491 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1036/2078] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:32.162691 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1043/2079] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:32 2022] PUT /identity/v3/projects/a687a9ada4494ab4b93aa989fe7b47a2/users/c44dad19055447d9a24206c441e2a0ab/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:32.169048 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52217f2-8815-4f2d-a3d1-1a2ce3761e0f service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:32.192887 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52217f2-8815-4f2d-a3d1-1a2ce3761e0f service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:32.328791 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f40a3a-a542-4140-a1e8-0bb539e4d43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:32.329657 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f40a3a-a542-4140-a1e8-0bb539e4d43d admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:32.330084 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f40a3a-a542-4140-a1e8-0bb539e4d43d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:32.330502 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f40a3a-a542-4140-a1e8-0bb539e4d43d admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:32.331557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f40a3a-a542-4140-a1e8-0bb539e4d43d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:32.334071 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f40a3a-a542-4140-a1e8-0bb539e4d43d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:32.354667 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17731acf-f169-4006-a088-36eb3e5ac513 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:32.359865 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1037/2080] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:32 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:32.372023 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17731acf-f169-4006-a088-36eb3e5ac513 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:32.378258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01749565-ad4b-4db5-98f2-632b26f47ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:32.379942 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01749565-ad4b-4db5-98f2-632b26f47ef7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:32.379942 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01749565-ad4b-4db5-98f2-632b26f47ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:32.380198 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01749565-ad4b-4db5-98f2-632b26f47ef7 admin admin] PATH_INFO: `/v3/projects/a687a9ada4494ab4b93aa989fe7b47a2/users/c44dad19055447d9a24206c441e2a0ab/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:32.383457 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01749565-ad4b-4db5-98f2-632b26f47ef7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a687a9ada4494ab4b93aa989fe7b47a2, user_id=c44dad19055447d9a24206c441e2a0ab, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:32.387763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01749565-ad4b-4db5-98f2-632b26f47ef7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:32.406254 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1044/2081] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:32 2022] PUT /identity/v3/projects/a687a9ada4494ab4b93aa989fe7b47a2/users/c44dad19055447d9a24206c441e2a0ab/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:33.739881 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1942e67-20ac-4dc2-ac31-f0e5eb72072c None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:33.740743 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1942e67-20ac-4dc2-ac31-f0e5eb72072c None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:33.740743 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1942e67-20ac-4dc2-ac31-f0e5eb72072c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:33.788856 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-a1942e67-20ac-4dc2-ac31-f0e5eb72072c None None] MFA Rules not processed for user `c44dad19055447d9a24206c441e2a0ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:33.790288 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a1942e67-20ac-4dc2-ac31-f0e5eb72072c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:33.920992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1038/2082] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:33 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:33.942744 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01749565-ad4b-4db5-98f2-632b26f47ef7 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:34.058163 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01749565-ad4b-4db5-98f2-632b26f47ef7 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:34.062516 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f7616-e0c2-4771-b024-1ba5c4ff2d0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:34.063292 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f7616-e0c2-4771-b024-1ba5c4ff2d0f service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:34.063582 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f7616-e0c2-4771-b024-1ba5c4ff2d0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:34.063864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f7616-e0c2-4771-b024-1ba5c4ff2d0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:34.077208 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:34.143214 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f7616-e0c2-4771-b024-1ba5c4ff2d0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:34.144986 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f7616-e0c2-4771-b024-1ba5c4ff2d0f service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:34.159148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1045/2083] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:35.885378 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1942e67-20ac-4dc2-ac31-f0e5eb72072c None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:35.903141 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1942e67-20ac-4dc2-ac31-f0e5eb72072c None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:35.913481 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e527b-abb6-4617-b82d-4eff481b5a31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:35.914291 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e527b-abb6-4617-b82d-4eff481b5a31 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:35.914578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e527b-abb6-4617-b82d-4eff481b5a31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:35.914847 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e527b-abb6-4617-b82d-4eff481b5a31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:35.928561 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e527b-abb6-4617-b82d-4eff481b5a31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:35.930154 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e527b-abb6-4617-b82d-4eff481b5a31 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:35.944048 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1039/2084] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:35 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:36.504511 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f7616-e0c2-4771-b024-1ba5c4ff2d0f service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:36.508036 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0d0f7616-e0c2-4771-b024-1ba5c4ff2d0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:36.715778 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f7616-e0c2-4771-b024-1ba5c4ff2d0f service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:36.748317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cef700-264f-459e-bba7-43dc8071ef52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:36.749541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cef700-264f-459e-bba7-43dc8071ef52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:36.750415 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cef700-264f-459e-bba7-43dc8071ef52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:36.750845 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cef700-264f-459e-bba7-43dc8071ef52 admin admin] PATH_INFO: `/v3/users/60eee823638646488d84e78b28265053` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:36.753396 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cef700-264f-459e-bba7-43dc8071ef52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60eee823638646488d84e78b28265053)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:36.757504 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cef700-264f-459e-bba7-43dc8071ef52 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:36.848875 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f5cef700-264f-459e-bba7-43dc8071ef52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60eee823638646488d84e78b28265053'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:36.850584 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f5cef700-264f-459e-bba7-43dc8071ef52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60eee823638646488d84e78b28265053'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:36.872809 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f5cef700-264f-459e-bba7-43dc8071ef52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60eee823638646488d84e78b28265053'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:36.887841 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f5cef700-264f-459e-bba7-43dc8071ef52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60eee823638646488d84e78b28265053'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:36.887841 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f5cef700-264f-459e-bba7-43dc8071ef52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60eee823638646488d84e78b28265053'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:36.910117 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1046/2085] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:36 2022] DELETE /identity/v3/users/60eee823638646488d84e78b28265053 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:36.917188 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30191589-ccad-4bb6-8e25-5da2b05c7083 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:36.918071 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30191589-ccad-4bb6-8e25-5da2b05c7083 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:36.918346 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30191589-ccad-4bb6-8e25-5da2b05c7083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:36.952341 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-30191589-ccad-4bb6-8e25-5da2b05c7083 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:36.955239 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-30191589-ccad-4bb6-8e25-5da2b05c7083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:36.958943 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cef700-264f-459e-bba7-43dc8071ef52 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:36.961482 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f5cef700-264f-459e-bba7-43dc8071ef52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:36.962059 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:37.059234 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:37.083131 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1040/2086] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:37.102270 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30191589-ccad-4bb6-8e25-5da2b05c7083 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:37.105336 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-30191589-ccad-4bb6-8e25-5da2b05c7083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:37.132723 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cef700-264f-459e-bba7-43dc8071ef52 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:37.140600 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e338f28-b38b-4910-aa79-fedb0e957f5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:37.143211 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e338f28-b38b-4910-aa79-fedb0e957f5e service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:37.143983 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e338f28-b38b-4910-aa79-fedb0e957f5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:37.144598 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e338f28-b38b-4910-aa79-fedb0e957f5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:37.148474 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0e338f28-b38b-4910-aa79-fedb0e957f5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:37.261400 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30191589-ccad-4bb6-8e25-5da2b05c7083 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:37.269867 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585f7e7-334a-47ea-b0a4-6104493cbb16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:37.271270 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585f7e7-334a-47ea-b0a4-6104493cbb16 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:37.271923 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585f7e7-334a-47ea-b0a4-6104493cbb16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:37.272637 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585f7e7-334a-47ea-b0a4-6104493cbb16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:37.296485 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5585f7e7-334a-47ea-b0a4-6104493cbb16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:37.301541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5585f7e7-334a-47ea-b0a4-6104493cbb16 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:37.329236 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1041/2087] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:37.343968 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e338f28-b38b-4910-aa79-fedb0e957f5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:37.350629 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e338f28-b38b-4910-aa79-fedb0e957f5e service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:37.381522 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1047/2088] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:37.836919 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585f7e7-334a-47ea-b0a4-6104493cbb16 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:37.840907 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5585f7e7-334a-47ea-b0a4-6104493cbb16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:37.887702 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585f7e7-334a-47ea-b0a4-6104493cbb16 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:37.897130 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb8915-b138-4a8d-a7d7-7439676ab998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:37.898407 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb8915-b138-4a8d-a7d7-7439676ab998 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:37.898912 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb8915-b138-4a8d-a7d7-7439676ab998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:37.899403 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb8915-b138-4a8d-a7d7-7439676ab998 admin admin] PATH_INFO: `/v3/projects/aa24f81f21d44d1d8980883b9576bde7` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:37.902273 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adb8915-b138-4a8d-a7d7-7439676ab998 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa24f81f21d44d1d8980883b9576bde7)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:37.910929 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adb8915-b138-4a8d-a7d7-7439676ab998 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:38.113410 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-2adb8915-b138-4a8d-a7d7-7439676ab998 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa24f81f21d44d1d8980883b9576bde7', 'request_id': 'req-2adb8915-b138-4a8d-a7d7-7439676ab998'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:38.129900 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-2adb8915-b138-4a8d-a7d7-7439676ab998 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa24f81f21d44d1d8980883b9576bde7', 'request_id': 'req-2adb8915-b138-4a8d-a7d7-7439676ab998'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:38.145291 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-2adb8915-b138-4a8d-a7d7-7439676ab998 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa24f81f21d44d1d8980883b9576bde7', 'request_id': 'req-2adb8915-b138-4a8d-a7d7-7439676ab998'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:38.169124 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-2adb8915-b138-4a8d-a7d7-7439676ab998 admin admin] The token cache is being invalidate because project aa24f81f21d44d1d8980883b9576bde7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:38.169644 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-2adb8915-b138-4a8d-a7d7-7439676ab998 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:38.172975 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-2adb8915-b138-4a8d-a7d7-7439676ab998 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:38.175308 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1042/2089] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:37 2022] DELETE /identity/v3/projects/aa24f81f21d44d1d8980883b9576bde7 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:38.180861 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e338f28-b38b-4910-aa79-fedb0e957f5e service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:38.204324 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e338f28-b38b-4910-aa79-fedb0e957f5e service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:38.301048 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ed774-2fb2-4d52-8a0a-a705fb2d1746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:38.302204 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ed774-2fb2-4d52-8a0a-a705fb2d1746 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:38.302824 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ed774-2fb2-4d52-8a0a-a705fb2d1746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:38.302963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ed774-2fb2-4d52-8a0a-a705fb2d1746 admin admin] PATH_INFO: `/v3/users/4ab6f9e242f049da8b768b55aea8ca96` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:38.304608 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ed774-2fb2-4d52-8a0a-a705fb2d1746 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ab6f9e242f049da8b768b55aea8ca96)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:38.307110 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ed774-2fb2-4d52-8a0a-a705fb2d1746 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:38.385401 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f09ed774-2fb2-4d52-8a0a-a705fb2d1746 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ab6f9e242f049da8b768b55aea8ca96'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:38.386197 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f09ed774-2fb2-4d52-8a0a-a705fb2d1746 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ab6f9e242f049da8b768b55aea8ca96'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:38.403259 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f09ed774-2fb2-4d52-8a0a-a705fb2d1746 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ab6f9e242f049da8b768b55aea8ca96'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:38.430845 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f09ed774-2fb2-4d52-8a0a-a705fb2d1746 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ab6f9e242f049da8b768b55aea8ca96'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:38.430845 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-f09ed774-2fb2-4d52-8a0a-a705fb2d1746 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ab6f9e242f049da8b768b55aea8ca96'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:38.458838 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1048/2090] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:38 2022] DELETE /identity/v3/users/4ab6f9e242f049da8b768b55aea8ca96 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:38.984865 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb8915-b138-4a8d-a7d7-7439676ab998 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:38.987372 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-2adb8915-b138-4a8d-a7d7-7439676ab998 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:39.021038 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb8915-b138-4a8d-a7d7-7439676ab998 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:39.099347 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:39.165659 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e2f4f2-158f-4618-afc6-a22b4c3a65ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:39.166720 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2f4f2-158f-4618-afc6-a22b4c3a65ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:39.166720 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2f4f2-158f-4618-afc6-a22b4c3a65ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:39.167020 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2f4f2-158f-4618-afc6-a22b4c3a65ea admin admin] PATH_INFO: `/v3/projects/0426318b15c54e28a94aa100c0035118` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:39.168771 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e2f4f2-158f-4618-afc6-a22b4c3a65ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=0426318b15c54e28a94aa100c0035118)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:39.187012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e2f4f2-158f-4618-afc6-a22b4c3a65ea admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:39.270376 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e3e2f4f2-158f-4618-afc6-a22b4c3a65ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0426318b15c54e28a94aa100c0035118', 'request_id': 'req-e3e2f4f2-158f-4618-afc6-a22b4c3a65ea'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:39.286734 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e3e2f4f2-158f-4618-afc6-a22b4c3a65ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0426318b15c54e28a94aa100c0035118', 'request_id': 'req-e3e2f4f2-158f-4618-afc6-a22b4c3a65ea'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:39.299636 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e3e2f4f2-158f-4618-afc6-a22b4c3a65ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0426318b15c54e28a94aa100c0035118', 'request_id': 'req-e3e2f4f2-158f-4618-afc6-a22b4c3a65ea'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:39.314922 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e3e2f4f2-158f-4618-afc6-a22b4c3a65ea admin admin] The token cache is being invalidate because project 0426318b15c54e28a94aa100c0035118 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:39.315360 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e3e2f4f2-158f-4618-afc6-a22b4c3a65ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:39.316380 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e3e2f4f2-158f-4618-afc6-a22b4c3a65ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:39.318293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1043/2091] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:38 2022] DELETE /identity/v3/projects/0426318b15c54e28a94aa100c0035118 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:39.323704 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ed774-2fb2-4d52-8a0a-a705fb2d1746 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:39.327110 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f09ed774-2fb2-4d52-8a0a-a705fb2d1746 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:39.338721 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:39.344487 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ed774-2fb2-4d52-8a0a-a705fb2d1746 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:39.463604 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0f55b-6d21-43e7-8241-2afe1844d295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:39.464743 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0f55b-6d21-43e7-8241-2afe1844d295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:39.465205 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0f55b-6d21-43e7-8241-2afe1844d295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:39.465675 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0f55b-6d21-43e7-8241-2afe1844d295 admin admin] PATH_INFO: `/v3/users/172e341cc4c24c5990f18f7df2555cc4` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:39.467249 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba0f55b-6d21-43e7-8241-2afe1844d295 admin admin] RBAC: Authorizing `identity:delete_user(user_id=172e341cc4c24c5990f18f7df2555cc4)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:39.470740 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba0f55b-6d21-43e7-8241-2afe1844d295 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:39.567121 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3ba0f55b-6d21-43e7-8241-2afe1844d295 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '172e341cc4c24c5990f18f7df2555cc4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:39.568611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3ba0f55b-6d21-43e7-8241-2afe1844d295 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '172e341cc4c24c5990f18f7df2555cc4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:39.585980 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3ba0f55b-6d21-43e7-8241-2afe1844d295 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '172e341cc4c24c5990f18f7df2555cc4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:39.600555 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3ba0f55b-6d21-43e7-8241-2afe1844d295 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '172e341cc4c24c5990f18f7df2555cc4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:39.601674 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3ba0f55b-6d21-43e7-8241-2afe1844d295 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '172e341cc4c24c5990f18f7df2555cc4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:39.630497 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1049/2092] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:39 2022] DELETE /identity/v3/users/172e341cc4c24c5990f18f7df2555cc4 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:40.182885 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e2f4f2-158f-4618-afc6-a22b4c3a65ea admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:40.186245 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e3e2f4f2-158f-4618-afc6-a22b4c3a65ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:40.219849 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e2f4f2-158f-4618-afc6-a22b4c3a65ea admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:40.314367 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2204cb39-994b-4f11-be4b-e232f5d4f1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:40.315071 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204cb39-994b-4f11-be4b-e232f5d4f1b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:40.315549 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204cb39-994b-4f11-be4b-e232f5d4f1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:40.315748 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204cb39-994b-4f11-be4b-e232f5d4f1b9 admin admin] PATH_INFO: `/v3/projects/8ff164eed8be4d859de8f16d230575de` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:40.317839 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2204cb39-994b-4f11-be4b-e232f5d4f1b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ff164eed8be4d859de8f16d230575de)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:40.319925 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2204cb39-994b-4f11-be4b-e232f5d4f1b9 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:40.396351 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-2204cb39-994b-4f11-be4b-e232f5d4f1b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ff164eed8be4d859de8f16d230575de', 'request_id': 'req-2204cb39-994b-4f11-be4b-e232f5d4f1b9'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:40.411766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-2204cb39-994b-4f11-be4b-e232f5d4f1b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ff164eed8be4d859de8f16d230575de', 'request_id': 'req-2204cb39-994b-4f11-be4b-e232f5d4f1b9'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:40.426672 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-2204cb39-994b-4f11-be4b-e232f5d4f1b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ff164eed8be4d859de8f16d230575de', 'request_id': 'req-2204cb39-994b-4f11-be4b-e232f5d4f1b9'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:40.437068 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:40.451649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-2204cb39-994b-4f11-be4b-e232f5d4f1b9 admin admin] The token cache is being invalidate because project 8ff164eed8be4d859de8f16d230575de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:40.452889 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-2204cb39-994b-4f11-be4b-e232f5d4f1b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:40.454399 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-2204cb39-994b-4f11-be4b-e232f5d4f1b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:40.457799 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1044/2093] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:40 2022] DELETE /identity/v3/projects/8ff164eed8be4d859de8f16d230575de => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:40.465802 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7e959-16a0-4719-98c6-83b3d54319bc None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:40.466436 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7e959-16a0-4719-98c6-83b3d54319bc None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:40.466596 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7e959-16a0-4719-98c6-83b3d54319bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:40.494976 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-7ce7e959-16a0-4719-98c6-83b3d54319bc None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:40.497533 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7ce7e959-16a0-4719-98c6-83b3d54319bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:40.646848 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1050/2094] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:40.649456 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7042a3-348f-4020-8d2d-bf4f0211ce13 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:40.649934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7042a3-348f-4020-8d2d-bf4f0211ce13 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:40.650403 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7042a3-348f-4020-8d2d-bf4f0211ce13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:40.687464 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-ce7042a3-348f-4020-8d2d-bf4f0211ce13 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:40.689324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ce7042a3-348f-4020-8d2d-bf4f0211ce13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:40.711059 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1045/2095] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:40.717518 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce7e959-16a0-4719-98c6-83b3d54319bc None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:40.736711 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce7e959-16a0-4719-98c6-83b3d54319bc None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:40.746244 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6936e525-cee5-4556-ae59-254911214bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:40.747453 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936e525-cee5-4556-ae59-254911214bda admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:40.748006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936e525-cee5-4556-ae59-254911214bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:40.748614 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936e525-cee5-4556-ae59-254911214bda admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:40.750868 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6936e525-cee5-4556-ae59-254911214bda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:40.751577 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6936e525-cee5-4556-ae59-254911214bda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:40.754755 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6936e525-cee5-4556-ae59-254911214bda admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:40.769772 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7042a3-348f-4020-8d2d-bf4f0211ce13 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:40.772678 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ce7042a3-348f-4020-8d2d-bf4f0211ce13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:40.780812 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1051/2096] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:09:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:40.787987 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6936e525-cee5-4556-ae59-254911214bda admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:40.807312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7042a3-348f-4020-8d2d-bf4f0211ce13 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:40.811670 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6936e525-cee5-4556-ae59-254911214bda admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:40.820658 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5bda9-1086-4794-bc05-ccb50706bb30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:40.821490 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc67aef-0d49-427e-8dae-d8972ddc09fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:40.822190 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc67aef-0d49-427e-8dae-d8972ddc09fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:40.822471 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc67aef-0d49-427e-8dae-d8972ddc09fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:40.822747 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc67aef-0d49-427e-8dae-d8972ddc09fd admin admin] PATH_INFO: `/v3/users/240ab5affd3c4ae8a85ba4148428e607` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:40.824702 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5bda9-1086-4794-bc05-ccb50706bb30 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:40.826704 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5bda9-1086-4794-bc05-ccb50706bb30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:40.828255 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc67aef-0d49-427e-8dae-d8972ddc09fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=240ab5affd3c4ae8a85ba4148428e607)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:40.829578 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5bda9-1086-4794-bc05-ccb50706bb30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:40.829992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc67aef-0d49-427e-8dae-d8972ddc09fd admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:40.830599 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a5bda9-1086-4794-bc05-ccb50706bb30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:40.837931 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a5bda9-1086-4794-bc05-ccb50706bb30 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:40.867946 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:40.922780 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1052/2097] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:40 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:09:40.929384 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5bda9-1086-4794-bc05-ccb50706bb30 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:40.975763 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-dcc67aef-0d49-427e-8dae-d8972ddc09fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '240ab5affd3c4ae8a85ba4148428e607'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:40.976925 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-dcc67aef-0d49-427e-8dae-d8972ddc09fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '240ab5affd3c4ae8a85ba4148428e607'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:41.005147 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-dcc67aef-0d49-427e-8dae-d8972ddc09fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '240ab5affd3c4ae8a85ba4148428e607'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:41.028660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-dcc67aef-0d49-427e-8dae-d8972ddc09fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '240ab5affd3c4ae8a85ba4148428e607'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:41.094626 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-dcc67aef-0d49-427e-8dae-d8972ddc09fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '240ab5affd3c4ae8a85ba4148428e607'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:41.139296 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1046/2098] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:40 2022] DELETE /identity/v3/users/240ab5affd3c4ae8a85ba4148428e607 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:41.145392 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2c878-6617-424f-bfcb-ed7d0906f005 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:41.145848 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2c878-6617-424f-bfcb-ed7d0906f005 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:41.146284 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2c878-6617-424f-bfcb-ed7d0906f005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:41.155775 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5bda9-1086-4794-bc05-ccb50706bb30 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:41.196987 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-71e2c878-6617-424f-bfcb-ed7d0906f005 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:41.198258 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-71e2c878-6617-424f-bfcb-ed7d0906f005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:41.394357 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f2bc1-8b84-4d26-a885-3121982deded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:41.396203 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f2bc1-8b84-4d26-a885-3121982deded admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:41.396952 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f2bc1-8b84-4d26-a885-3121982deded admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:41.397935 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f2bc1-8b84-4d26-a885-3121982deded admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:41.399066 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f2bc1-8b84-4d26-a885-3121982deded admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:41.404620 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f2bc1-8b84-4d26-a885-3121982deded admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:41.424412 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1047/2099] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:41.446088 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2c878-6617-424f-bfcb-ed7d0906f005 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:41.447720 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-71e2c878-6617-424f-bfcb-ed7d0906f005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:41.450851 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1053/2100] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:40 2022] POST /identity/v3/users => generated 293 bytes in 523 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:09:41.458745 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f2bc1-8b84-4d26-a885-3121982deded admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:41.461597 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e90f2bc1-8b84-4d26-a885-3121982deded admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:41.481470 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f2bc1-8b84-4d26-a885-3121982deded admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:41.489635 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fe6a10-56a2-418f-8457-a10d628e712b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:41.490872 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe6a10-56a2-418f-8457-a10d628e712b admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:41.491322 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe6a10-56a2-418f-8457-a10d628e712b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:41.491779 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe6a10-56a2-418f-8457-a10d628e712b admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:41.493500 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fe6a10-56a2-418f-8457-a10d628e712b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:41.496071 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fe6a10-56a2-418f-8457-a10d628e712b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:41.516049 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1054/2101] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:41.519913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fe6a10-56a2-418f-8457-a10d628e712b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:41.537564 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fe6a10-56a2-418f-8457-a10d628e712b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:41.550501 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54b955-fad1-47d2-a4b0-59c2e9334c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:41.551699 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54b955-fad1-47d2-a4b0-59c2e9334c22 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:41.552344 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54b955-fad1-47d2-a4b0-59c2e9334c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:41.552922 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54b955-fad1-47d2-a4b0-59c2e9334c22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:41.554618 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e54b955-fad1-47d2-a4b0-59c2e9334c22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:41.558280 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e54b955-fad1-47d2-a4b0-59c2e9334c22 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:41.586633 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1055/2102] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:41.589721 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2c878-6617-424f-bfcb-ed7d0906f005 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:41.597209 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54b955-fad1-47d2-a4b0-59c2e9334c22 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:41.600289 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12991d40-e311-4c32-88e4-3171040f070a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:41.602719 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12991d40-e311-4c32-88e4-3171040f070a service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:41.602719 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12991d40-e311-4c32-88e4-3171040f070a service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:41.602719 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12991d40-e311-4c32-88e4-3171040f070a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:41.619417 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54b955-fad1-47d2-a4b0-59c2e9334c22 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:41.623133 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12991d40-e311-4c32-88e4-3171040f070a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:41.625934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12991d40-e311-4c32-88e4-3171040f070a service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:41.629216 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d5e4d-f4fe-43ae-aa0e-423bee5c03b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:41.630683 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d5e4d-f4fe-43ae-aa0e-423bee5c03b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:41.631302 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d5e4d-f4fe-43ae-aa0e-423bee5c03b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:41.631926 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d5e4d-f4fe-43ae-aa0e-423bee5c03b4 admin admin] PATH_INFO: `/v3/projects/9d78f1cca2f845a5ad90c514c2d71f62/users/ec40d75e772442d2a97e9bf18b4b90c8/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:41.649766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1048/2103] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:41.665930 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d5e4d-f4fe-43ae-aa0e-423bee5c03b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d78f1cca2f845a5ad90c514c2d71f62, user_id=ec40d75e772442d2a97e9bf18b4b90c8, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:41.672003 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d5e4d-f4fe-43ae-aa0e-423bee5c03b4 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:41.696847 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1056/2104] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:41 2022] PUT /identity/v3/projects/9d78f1cca2f845a5ad90c514c2d71f62/users/ec40d75e772442d2a97e9bf18b4b90c8/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:41.705935 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc37f84-1ff1-4e83-9f58-acf3f01cbe6a None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:41.706586 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc37f84-1ff1-4e83-9f58-acf3f01cbe6a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:41.707048 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc37f84-1ff1-4e83-9f58-acf3f01cbe6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:41.717838 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:41.735765 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-5cc37f84-1ff1-4e83-9f58-acf3f01cbe6a None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:41.740094 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5cc37f84-1ff1-4e83-9f58-acf3f01cbe6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:41.849648 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1049/2105] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:41.863884 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d5e4d-f4fe-43ae-aa0e-423bee5c03b4 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:41.867517 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-bc8d5e4d-f4fe-43ae-aa0e-423bee5c03b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:41.944261 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:41.973970 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d5e4d-f4fe-43ae-aa0e-423bee5c03b4 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:41.979255 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9249810c-9494-48a2-aece-b4718b68ecd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:41.980149 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249810c-9494-48a2-aece-b4718b68ecd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:41.980594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249810c-9494-48a2-aece-b4718b68ecd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:41.981127 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249810c-9494-48a2-aece-b4718b68ecd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:42.001675 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9249810c-9494-48a2-aece-b4718b68ecd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:42.003505 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9249810c-9494-48a2-aece-b4718b68ecd9 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:42.021475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1057/2106] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:42.456395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc37f84-1ff1-4e83-9f58-acf3f01cbe6a None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:42.465632 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5cc37f84-1ff1-4e83-9f58-acf3f01cbe6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:42.491178 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc37f84-1ff1-4e83-9f58-acf3f01cbe6a None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:42.498418 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c8f45-bbfc-4c50-a07f-be4b17813e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:42.499504 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c8f45-bbfc-4c50-a07f-be4b17813e9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:42.500338 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c8f45-bbfc-4c50-a07f-be4b17813e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:42.500810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c8f45-bbfc-4c50-a07f-be4b17813e9a admin admin] PATH_INFO: `/v3/projects/f812076f1abf4f03a2708691c93560b1` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:42.502854 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818c8f45-bbfc-4c50-a07f-be4b17813e9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f812076f1abf4f03a2708691c93560b1)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:42.505164 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818c8f45-bbfc-4c50-a07f-be4b17813e9a admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:42.606236 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-818c8f45-bbfc-4c50-a07f-be4b17813e9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f812076f1abf4f03a2708691c93560b1', 'request_id': 'req-818c8f45-bbfc-4c50-a07f-be4b17813e9a'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:42.622032 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-818c8f45-bbfc-4c50-a07f-be4b17813e9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f812076f1abf4f03a2708691c93560b1', 'request_id': 'req-818c8f45-bbfc-4c50-a07f-be4b17813e9a'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:42.642603 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-818c8f45-bbfc-4c50-a07f-be4b17813e9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f812076f1abf4f03a2708691c93560b1', 'request_id': 'req-818c8f45-bbfc-4c50-a07f-be4b17813e9a'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:42.713997 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-818c8f45-bbfc-4c50-a07f-be4b17813e9a admin admin] The token cache is being invalidate because project f812076f1abf4f03a2708691c93560b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:42.714495 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-818c8f45-bbfc-4c50-a07f-be4b17813e9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:42.716691 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-818c8f45-bbfc-4c50-a07f-be4b17813e9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:42.718757 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1050/2107] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:42 2022] DELETE /identity/v3/projects/f812076f1abf4f03a2708691c93560b1 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:42.724786 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9249810c-9494-48a2-aece-b4718b68ecd9 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:42.750728 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9249810c-9494-48a2-aece-b4718b68ecd9 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:42.941936 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff45e3e7-c2dd-4527-a61f-43c3d826269d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:42.942854 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45e3e7-c2dd-4527-a61f-43c3d826269d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:42.943321 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45e3e7-c2dd-4527-a61f-43c3d826269d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:42.943816 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45e3e7-c2dd-4527-a61f-43c3d826269d admin admin] PATH_INFO: `/v3/users/b3875fbc3cbc4ee19a89a79cdd6c2753` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:42.946368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff45e3e7-c2dd-4527-a61f-43c3d826269d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3875fbc3cbc4ee19a89a79cdd6c2753)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:42.950834 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff45e3e7-c2dd-4527-a61f-43c3d826269d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:43.088061 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ff45e3e7-c2dd-4527-a61f-43c3d826269d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3875fbc3cbc4ee19a89a79cdd6c2753'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:43.089115 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ff45e3e7-c2dd-4527-a61f-43c3d826269d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3875fbc3cbc4ee19a89a79cdd6c2753'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:43.206788 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ff45e3e7-c2dd-4527-a61f-43c3d826269d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3875fbc3cbc4ee19a89a79cdd6c2753'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:43.230307 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ff45e3e7-c2dd-4527-a61f-43c3d826269d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3875fbc3cbc4ee19a89a79cdd6c2753'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:43.231628 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ff45e3e7-c2dd-4527-a61f-43c3d826269d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3875fbc3cbc4ee19a89a79cdd6c2753'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:43.283133 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1058/2108] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:42 2022] DELETE /identity/v3/users/b3875fbc3cbc4ee19a89a79cdd6c2753 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:44.294704 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c8f45-bbfc-4c50-a07f-be4b17813e9a admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:44.298377 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-818c8f45-bbfc-4c50-a07f-be4b17813e9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:44.353585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c8f45-bbfc-4c50-a07f-be4b17813e9a admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:44.668938 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08525078-b5c3-4054-b9ad-b2a450e5f112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:44.672796 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08525078-b5c3-4054-b9ad-b2a450e5f112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:44.673197 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08525078-b5c3-4054-b9ad-b2a450e5f112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:44.673431 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08525078-b5c3-4054-b9ad-b2a450e5f112 admin admin] PATH_INFO: `/v3/projects/6636e1a220944e0ea518b444ffc5b784` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:44.678889 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08525078-b5c3-4054-b9ad-b2a450e5f112 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6636e1a220944e0ea518b444ffc5b784)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:44.692214 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08525078-b5c3-4054-b9ad-b2a450e5f112 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:44.879548 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:44.964571 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-08525078-b5c3-4054-b9ad-b2a450e5f112 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6636e1a220944e0ea518b444ffc5b784', 'request_id': 'req-08525078-b5c3-4054-b9ad-b2a450e5f112'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:44.981467 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-08525078-b5c3-4054-b9ad-b2a450e5f112 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6636e1a220944e0ea518b444ffc5b784', 'request_id': 'req-08525078-b5c3-4054-b9ad-b2a450e5f112'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:45.006270 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-08525078-b5c3-4054-b9ad-b2a450e5f112 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6636e1a220944e0ea518b444ffc5b784', 'request_id': 'req-08525078-b5c3-4054-b9ad-b2a450e5f112'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:45.033869 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-08525078-b5c3-4054-b9ad-b2a450e5f112 admin admin] The token cache is being invalidate because project 6636e1a220944e0ea518b444ffc5b784 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:45.035720 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-08525078-b5c3-4054-b9ad-b2a450e5f112 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:45.038348 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-08525078-b5c3-4054-b9ad-b2a450e5f112 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:45.048510 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1051/2109] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:44 2022] DELETE /identity/v3/projects/6636e1a220944e0ea518b444ffc5b784 => generated 0 bytes in 752 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:45.050168 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff45e3e7-c2dd-4527-a61f-43c3d826269d admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:45.053496 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ff45e3e7-c2dd-4527-a61f-43c3d826269d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:45.127949 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff45e3e7-c2dd-4527-a61f-43c3d826269d admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:45.394633 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e78deba-b9a3-4df6-9e14-54dc6c2ebc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:45.395789 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78deba-b9a3-4df6-9e14-54dc6c2ebc13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:45.396263 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78deba-b9a3-4df6-9e14-54dc6c2ebc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:45.396740 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78deba-b9a3-4df6-9e14-54dc6c2ebc13 admin admin] PATH_INFO: `/v3/users/fff9c5fa618e4f65b24b052704041221` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:45.400183 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e78deba-b9a3-4df6-9e14-54dc6c2ebc13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fff9c5fa618e4f65b24b052704041221)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:45.402546 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e78deba-b9a3-4df6-9e14-54dc6c2ebc13 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:45.411114 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:45.581036 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2e78deba-b9a3-4df6-9e14-54dc6c2ebc13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fff9c5fa618e4f65b24b052704041221'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:45.582618 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2e78deba-b9a3-4df6-9e14-54dc6c2ebc13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fff9c5fa618e4f65b24b052704041221'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:45.642705 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2e78deba-b9a3-4df6-9e14-54dc6c2ebc13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fff9c5fa618e4f65b24b052704041221'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:45.658889 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2e78deba-b9a3-4df6-9e14-54dc6c2ebc13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fff9c5fa618e4f65b24b052704041221'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:45.659905 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2e78deba-b9a3-4df6-9e14-54dc6c2ebc13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fff9c5fa618e4f65b24b052704041221'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:45.693053 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1059/2110] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:45 2022] DELETE /identity/v3/users/fff9c5fa618e4f65b24b052704041221 => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:46.757891 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08525078-b5c3-4054-b9ad-b2a450e5f112 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:46.760193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-08525078-b5c3-4054-b9ad-b2a450e5f112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:46.875029 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08525078-b5c3-4054-b9ad-b2a450e5f112 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:47.039579 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fe5160-92b9-4bf9-a3e6-90a65b2e7cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:47.041732 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe5160-92b9-4bf9-a3e6-90a65b2e7cb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:47.042392 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe5160-92b9-4bf9-a3e6-90a65b2e7cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:47.043912 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe5160-92b9-4bf9-a3e6-90a65b2e7cb9 admin admin] PATH_INFO: `/v3/projects/32eeebdbc7d64f319568c7808cf620d4` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:47.046475 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fe5160-92b9-4bf9-a3e6-90a65b2e7cb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32eeebdbc7d64f319568c7808cf620d4)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:47.052959 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fe5160-92b9-4bf9-a3e6-90a65b2e7cb9 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:47.195495 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-99fe5160-92b9-4bf9-a3e6-90a65b2e7cb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32eeebdbc7d64f319568c7808cf620d4', 'request_id': 'req-99fe5160-92b9-4bf9-a3e6-90a65b2e7cb9'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:47.206513 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-99fe5160-92b9-4bf9-a3e6-90a65b2e7cb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32eeebdbc7d64f319568c7808cf620d4', 'request_id': 'req-99fe5160-92b9-4bf9-a3e6-90a65b2e7cb9'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:47.223162 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-99fe5160-92b9-4bf9-a3e6-90a65b2e7cb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32eeebdbc7d64f319568c7808cf620d4', 'request_id': 'req-99fe5160-92b9-4bf9-a3e6-90a65b2e7cb9'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:47.256679 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-99fe5160-92b9-4bf9-a3e6-90a65b2e7cb9 admin admin] The token cache is being invalidate because project 32eeebdbc7d64f319568c7808cf620d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:47.257277 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-99fe5160-92b9-4bf9-a3e6-90a65b2e7cb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:47.258192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-99fe5160-92b9-4bf9-a3e6-90a65b2e7cb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:47.260308 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1052/2111] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:46 2022] DELETE /identity/v3/projects/32eeebdbc7d64f319568c7808cf620d4 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:47.278153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc0b5f-3f7d-4c0c-8637-6dde8e464b7c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:47.279519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc0b5f-3f7d-4c0c-8637-6dde8e464b7c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:47.280071 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc0b5f-3f7d-4c0c-8637-6dde8e464b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:47.317290 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-75cc0b5f-3f7d-4c0c-8637-6dde8e464b7c None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:47.319539 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-75cc0b5f-3f7d-4c0c-8637-6dde8e464b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:47.467389 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1060/2112] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:47.489408 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cbb355-2d75-4f5e-a4d1-a16a5a7d7a9b None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:47.490082 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cbb355-2d75-4f5e-a4d1-a16a5a7d7a9b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:47.492084 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cbb355-2d75-4f5e-a4d1-a16a5a7d7a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:47.536386 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-b3cbb355-2d75-4f5e-a4d1-a16a5a7d7a9b None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:47.538438 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b3cbb355-2d75-4f5e-a4d1-a16a5a7d7a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:47.567243 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1053/2113] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:47.574579 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc0b5f-3f7d-4c0c-8637-6dde8e464b7c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:47.593716 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc0b5f-3f7d-4c0c-8637-6dde8e464b7c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:47.623419 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0143dc-4356-4aba-ae99-ab039c11506f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:47.624822 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0143dc-4356-4aba-ae99-ab039c11506f admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:47.625465 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0143dc-4356-4aba-ae99-ab039c11506f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:47.625855 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0143dc-4356-4aba-ae99-ab039c11506f admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:47.627610 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0143dc-4356-4aba-ae99-ab039c11506f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:47.630766 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0143dc-4356-4aba-ae99-ab039c11506f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:47.635152 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0143dc-4356-4aba-ae99-ab039c11506f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:47.675996 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1061/2114] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:09:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:09:47.681196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cbb355-2d75-4f5e-a4d1-a16a5a7d7a9b None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:47.691882 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cbb355-2d75-4f5e-a4d1-a16a5a7d7a9b None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:47.697713 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774de8c-d1df-472c-88e3-b63c89f6ebfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:47.698379 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774de8c-d1df-472c-88e3-b63c89f6ebfd admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:47.698659 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774de8c-d1df-472c-88e3-b63c89f6ebfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:47.698931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774de8c-d1df-472c-88e3-b63c89f6ebfd admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:47.699631 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5774de8c-d1df-472c-88e3-b63c89f6ebfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:47.701972 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5774de8c-d1df-472c-88e3-b63c89f6ebfd admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:47.758287 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1054/2115] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:47 2022] POST /identity/v3/projects => generated 385 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:47.759320 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0143dc-4356-4aba-ae99-ab039c11506f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:47.786486 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0143dc-4356-4aba-ae99-ab039c11506f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:47.926753 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e962b8-f13d-48fe-8587-1f6c61a74c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:47.927498 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e962b8-f13d-48fe-8587-1f6c61a74c19 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:47.928070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e962b8-f13d-48fe-8587-1f6c61a74c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:47.929596 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e962b8-f13d-48fe-8587-1f6c61a74c19 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:47.930170 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e962b8-f13d-48fe-8587-1f6c61a74c19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:47.933713 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e962b8-f13d-48fe-8587-1f6c61a74c19 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:47.973738 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:47.993997 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1062/2116] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:47 2022] POST /identity/v3/users => generated 293 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:47.995754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774de8c-d1df-472c-88e3-b63c89f6ebfd admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:48.011176 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774de8c-d1df-472c-88e3-b63c89f6ebfd admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:48.025696 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cabf8a0-992d-4fca-a2d8-e073c49a6fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:48.026352 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabf8a0-992d-4fca-a2d8-e073c49a6fa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:48.026812 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabf8a0-992d-4fca-a2d8-e073c49a6fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:48.027245 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabf8a0-992d-4fca-a2d8-e073c49a6fa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:48.028151 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cabf8a0-992d-4fca-a2d8-e073c49a6fa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:48.031101 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cabf8a0-992d-4fca-a2d8-e073c49a6fa4 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:48.052514 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1055/2117] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:47 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:48.055788 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e962b8-f13d-48fe-8587-1f6c61a74c19 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:48.068602 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e962b8-f13d-48fe-8587-1f6c61a74c19 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:48.074839 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c67ab2-489d-4656-aa99-3c1907b66114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:48.075572 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c67ab2-489d-4656-aa99-3c1907b66114 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:48.075920 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c67ab2-489d-4656-aa99-3c1907b66114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:48.076323 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c67ab2-489d-4656-aa99-3c1907b66114 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:48.077114 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c67ab2-489d-4656-aa99-3c1907b66114 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:48.079147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c67ab2-489d-4656-aa99-3c1907b66114 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:48.093288 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1063/2118] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:48.097891 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cabf8a0-992d-4fca-a2d8-e073c49a6fa4 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:48.110977 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cabf8a0-992d-4fca-a2d8-e073c49a6fa4 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:48.117083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3539b491-5890-442e-ae57-bbef381564c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:48.117880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539b491-5890-442e-ae57-bbef381564c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:48.118196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539b491-5890-442e-ae57-bbef381564c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:48.118462 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539b491-5890-442e-ae57-bbef381564c7 admin admin] PATH_INFO: `/v3/projects/6a2e679e5e5e415284ead8a579ad3dd3/users/099d463ec7bc43a7a868e785cfdf659b/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:48.125309 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:48.149600 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3539b491-5890-442e-ae57-bbef381564c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a2e679e5e5e415284ead8a579ad3dd3, user_id=099d463ec7bc43a7a868e785cfdf659b, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:48.156147 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3539b491-5890-442e-ae57-bbef381564c7 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:48.182371 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1056/2119] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:48 2022] PUT /identity/v3/projects/6a2e679e5e5e415284ead8a579ad3dd3/users/099d463ec7bc43a7a868e785cfdf659b/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:48.197267 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438c32c-ac9b-4866-8227-1dfc9caef1bb None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:48.197558 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438c32c-ac9b-4866-8227-1dfc9caef1bb None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:48.198670 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438c32c-ac9b-4866-8227-1dfc9caef1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:48.276689 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b438c32c-ac9b-4866-8227-1dfc9caef1bb None None] MFA Rules not processed for user `099d463ec7bc43a7a868e785cfdf659b`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:48.278071 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b438c32c-ac9b-4866-8227-1dfc9caef1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:48.521097 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1064/2120] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:48.568001 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3539b491-5890-442e-ae57-bbef381564c7 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:48.578915 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3539b491-5890-442e-ae57-bbef381564c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:48.811151 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3539b491-5890-442e-ae57-bbef381564c7 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:48.817902 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eca59d-ae0a-41fc-acc3-d2c351e89c92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:48.818979 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eca59d-ae0a-41fc-acc3-d2c351e89c92 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:48.819714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eca59d-ae0a-41fc-acc3-d2c351e89c92 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:48.820597 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eca59d-ae0a-41fc-acc3-d2c351e89c92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:48.837972 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eca59d-ae0a-41fc-acc3-d2c351e89c92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:48.840440 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eca59d-ae0a-41fc-acc3-d2c351e89c92 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:48.862954 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1057/2121] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:49.616607 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b438c32c-ac9b-4866-8227-1dfc9caef1bb None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:49.619227 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b438c32c-ac9b-4866-8227-1dfc9caef1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:49.731099 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b438c32c-ac9b-4866-8227-1dfc9caef1bb None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:49.738328 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365be24d-bbc3-4abf-a7b3-3dba159be3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:49.739203 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365be24d-bbc3-4abf-a7b3-3dba159be3ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:49.739591 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365be24d-bbc3-4abf-a7b3-3dba159be3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:49.740006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365be24d-bbc3-4abf-a7b3-3dba159be3ff admin admin] PATH_INFO: `/v3/users/1bc63b9c6f7d424a91c72c1f44e5c8ef` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:49.741917 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365be24d-bbc3-4abf-a7b3-3dba159be3ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bc63b9c6f7d424a91c72c1f44e5c8ef)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:49.745264 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365be24d-bbc3-4abf-a7b3-3dba159be3ff admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:49.825094 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-365be24d-bbc3-4abf-a7b3-3dba159be3ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bc63b9c6f7d424a91c72c1f44e5c8ef'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:49.826288 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-365be24d-bbc3-4abf-a7b3-3dba159be3ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bc63b9c6f7d424a91c72c1f44e5c8ef'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:49.845522 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-365be24d-bbc3-4abf-a7b3-3dba159be3ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bc63b9c6f7d424a91c72c1f44e5c8ef'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:49.856449 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-365be24d-bbc3-4abf-a7b3-3dba159be3ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bc63b9c6f7d424a91c72c1f44e5c8ef'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:49.857723 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-365be24d-bbc3-4abf-a7b3-3dba159be3ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bc63b9c6f7d424a91c72c1f44e5c8ef'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:49.865681 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:49.891355 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1065/2122] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:49 2022] DELETE /identity/v3/users/1bc63b9c6f7d424a91c72c1f44e5c8ef => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:49.898929 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16509861-d6b8-4063-be38-9962f8b7dd0b None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:49.899980 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16509861-d6b8-4063-be38-9962f8b7dd0b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:49.900519 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16509861-d6b8-4063-be38-9962f8b7dd0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:49.935636 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-16509861-d6b8-4063-be38-9962f8b7dd0b None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:49.937725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-16509861-d6b8-4063-be38-9962f8b7dd0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:50.122397 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1058/2123] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:50.145845 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365be24d-bbc3-4abf-a7b3-3dba159be3ff admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:50.147744 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-365be24d-bbc3-4abf-a7b3-3dba159be3ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:50.308732 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad5170-ce33-42fd-95e4-937cc80bd0e6 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:50.309595 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad5170-ce33-42fd-95e4-937cc80bd0e6 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:50.310497 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad5170-ce33-42fd-95e4-937cc80bd0e6 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:50.311941 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365be24d-bbc3-4abf-a7b3-3dba159be3ff admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:50.313699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1059/2124] 10.222.0.45 () {54 vars in 921 bytes} [Mon Aug 1 14:09:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:09:50.323098 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4ec5e-eae1-4c2e-afb3-db1635c8645d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:50.324120 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4ec5e-eae1-4c2e-afb3-db1635c8645d service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:50.324697 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4ec5e-eae1-4c2e-afb3-db1635c8645d service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:50.325156 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4ec5e-eae1-4c2e-afb3-db1635c8645d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:50.351207 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b338f-4c0d-46b1-978f-d6620366c344 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:50.352005 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b338f-4c0d-46b1-978f-d6620366c344 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:50.352771 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b338f-4c0d-46b1-978f-d6620366c344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:50.355765 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae4ec5e-eae1-4c2e-afb3-db1635c8645d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:50.360836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae4ec5e-eae1-4c2e-afb3-db1635c8645d service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:50.397072 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1066/2125] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:50.421639 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-832b338f-4c0d-46b1-978f-d6620366c344 None None] MFA Rules not processed for user `58380e30789041e4ad9e8652bc628658`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:50.423980 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-832b338f-4c0d-46b1-978f-d6620366c344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:50.485835 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4ec5e-eae1-4c2e-afb3-db1635c8645d service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:50.489908 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6ae4ec5e-eae1-4c2e-afb3-db1635c8645d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:50.539118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4ec5e-eae1-4c2e-afb3-db1635c8645d service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:50.561232 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4afecb-ed63-421c-b225-673b6137ca4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:50.562093 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4afecb-ed63-421c-b225-673b6137ca4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:50.562816 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4afecb-ed63-421c-b225-673b6137ca4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:50.563112 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4afecb-ed63-421c-b225-673b6137ca4a admin admin] PATH_INFO: `/v3/users/ed6e0b24487342c1a9246579c0854653` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:50.566557 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4afecb-ed63-421c-b225-673b6137ca4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed6e0b24487342c1a9246579c0854653)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:50.573451 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4afecb-ed63-421c-b225-673b6137ca4a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:50.712325 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1060/2126] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Aug 1 14:09:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:50.738435 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832b338f-4c0d-46b1-978f-d6620366c344 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:50.742099 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-832b338f-4c0d-46b1-978f-d6620366c344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:50.790139 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832b338f-4c0d-46b1-978f-d6620366c344 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:50.797107 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1c4afecb-ed63-421c-b225-673b6137ca4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed6e0b24487342c1a9246579c0854653'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:50.799071 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662da82-25f4-42dc-ac6c-e83979961442 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58380e30789041e4ad9e8652bc628658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:50.800016 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662da82-25f4-42dc-ac6c-e83979961442 service placement] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:50.800599 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662da82-25f4-42dc-ac6c-e83979961442 service placement] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:50.801046 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1c4afecb-ed63-421c-b225-673b6137ca4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed6e0b24487342c1a9246579c0854653'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:50.801408 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662da82-25f4-42dc-ac6c-e83979961442 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:50.815204 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3662da82-25f4-42dc-ac6c-e83979961442 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:50.817366 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3662da82-25f4-42dc-ac6c-e83979961442 service placement] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:50.844599 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1c4afecb-ed63-421c-b225-673b6137ca4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed6e0b24487342c1a9246579c0854653'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:50.896360 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1061/2127] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:09:50.898023 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1c4afecb-ed63-421c-b225-673b6137ca4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed6e0b24487342c1a9246579c0854653'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:50.902471 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1c4afecb-ed63-421c-b225-673b6137ca4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed6e0b24487342c1a9246579c0854653'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:50.952605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1067/2128] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:50 2022] DELETE /identity/v3/users/ed6e0b24487342c1a9246579c0854653 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:51.374293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662da82-25f4-42dc-ac6c-e83979961442 service placement] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:51.380557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3662da82-25f4-42dc-ac6c-e83979961442 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:51.437887 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662da82-25f4-42dc-ac6c-e83979961442 service placement] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:51.450760 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:51.625893 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f83b2d-3a48-484b-8fba-e207cb7bbc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:51.629710 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f83b2d-3a48-484b-8fba-e207cb7bbc99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:51.634881 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f83b2d-3a48-484b-8fba-e207cb7bbc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:51.634881 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f83b2d-3a48-484b-8fba-e207cb7bbc99 admin admin] PATH_INFO: `/v3/projects/f1674f04a12949ca92c7b519811b2b4c` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:51.649542 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f83b2d-3a48-484b-8fba-e207cb7bbc99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1674f04a12949ca92c7b519811b2b4c)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:51.656355 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f83b2d-3a48-484b-8fba-e207cb7bbc99 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:51.735122 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4afecb-ed63-421c-b225-673b6137ca4a admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:51.739386 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1c4afecb-ed63-421c-b225-673b6137ca4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:51.819267 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4afecb-ed63-421c-b225-673b6137ca4a admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:51.865059 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d35fa-4103-4879-b29a-e98b0f4d1fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:51.870632 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d35fa-4103-4879-b29a-e98b0f4d1fd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:51.871866 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d35fa-4103-4879-b29a-e98b0f4d1fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:51.874437 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d35fa-4103-4879-b29a-e98b0f4d1fd4 admin admin] PATH_INFO: `/v3/projects/075e439a5de54261b5357fe8c513e9ee` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:51.883300 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0d35fa-4103-4879-b29a-e98b0f4d1fd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=075e439a5de54261b5357fe8c513e9ee)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:51.901170 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0d35fa-4103-4879-b29a-e98b0f4d1fd4 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:51.916253 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b5f83b2d-3a48-484b-8fba-e207cb7bbc99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1674f04a12949ca92c7b519811b2b4c', 'request_id': 'req-b5f83b2d-3a48-484b-8fba-e207cb7bbc99'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:51.932329 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b5f83b2d-3a48-484b-8fba-e207cb7bbc99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1674f04a12949ca92c7b519811b2b4c', 'request_id': 'req-b5f83b2d-3a48-484b-8fba-e207cb7bbc99'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:51.970729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b5f83b2d-3a48-484b-8fba-e207cb7bbc99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1674f04a12949ca92c7b519811b2b4c', 'request_id': 'req-b5f83b2d-3a48-484b-8fba-e207cb7bbc99'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:52.026696 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b5f83b2d-3a48-484b-8fba-e207cb7bbc99 admin admin] The token cache is being invalidate because project f1674f04a12949ca92c7b519811b2b4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:52.027555 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b5f83b2d-3a48-484b-8fba-e207cb7bbc99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:52.028992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b5f83b2d-3a48-484b-8fba-e207cb7bbc99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:52.052115 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1062/2129] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:51 2022] DELETE /identity/v3/projects/f1674f04a12949ca92c7b519811b2b4c => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:52.054698 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31598fe-7b61-4bb2-a6bc-1831b7871c38 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:52.055157 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31598fe-7b61-4bb2-a6bc-1831b7871c38 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:52.055627 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31598fe-7b61-4bb2-a6bc-1831b7871c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:52.098442 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-c31598fe-7b61-4bb2-a6bc-1831b7871c38 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:52.100740 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c31598fe-7b61-4bb2-a6bc-1831b7871c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:52.134227 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fa0d35fa-4103-4879-b29a-e98b0f4d1fd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '075e439a5de54261b5357fe8c513e9ee', 'request_id': 'req-fa0d35fa-4103-4879-b29a-e98b0f4d1fd4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:52.157663 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fa0d35fa-4103-4879-b29a-e98b0f4d1fd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '075e439a5de54261b5357fe8c513e9ee', 'request_id': 'req-fa0d35fa-4103-4879-b29a-e98b0f4d1fd4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:52.186813 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fa0d35fa-4103-4879-b29a-e98b0f4d1fd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '075e439a5de54261b5357fe8c513e9ee', 'request_id': 'req-fa0d35fa-4103-4879-b29a-e98b0f4d1fd4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:52.190722 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:52.246431 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1063/2130] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:52.257205 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699608ce-93dc-4068-9e91-111226fe1b93 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:52.257483 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699608ce-93dc-4068-9e91-111226fe1b93 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:52.257762 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699608ce-93dc-4068-9e91-111226fe1b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:52.270073 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fa0d35fa-4103-4879-b29a-e98b0f4d1fd4 admin admin] The token cache is being invalidate because project 075e439a5de54261b5357fe8c513e9ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:52.270577 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fa0d35fa-4103-4879-b29a-e98b0f4d1fd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:52.276420 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fa0d35fa-4103-4879-b29a-e98b0f4d1fd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:52.283050 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-699608ce-93dc-4068-9e91-111226fe1b93 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:52.283446 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1068/2131] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:51 2022] DELETE /identity/v3/projects/075e439a5de54261b5357fe8c513e9ee => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:52.285212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-699608ce-93dc-4068-9e91-111226fe1b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:52.292789 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74f8f4-9f08-45d2-b2de-3eacc09cf5f5 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:52.293381 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74f8f4-9f08-45d2-b2de-3eacc09cf5f5 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:52.293959 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74f8f4-9f08-45d2-b2de-3eacc09cf5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:52.315092 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1064/2132] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:52.316672 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699608ce-93dc-4068-9e91-111226fe1b93 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:52.328158 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699608ce-93dc-4068-9e91-111226fe1b93 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:52.336941 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be2298-3597-4fb7-9892-92fa3538b881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:52.337783 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be2298-3597-4fb7-9892-92fa3538b881 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:52.338138 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be2298-3597-4fb7-9892-92fa3538b881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:52.347733 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be2298-3597-4fb7-9892-92fa3538b881 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:52.347733 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be2298-3597-4fb7-9892-92fa3538b881 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:52.347733 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be2298-3597-4fb7-9892-92fa3538b881 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:52.347733 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be2298-3597-4fb7-9892-92fa3538b881 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:52.361284 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1065/2133] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:09:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:52.366969 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be2298-3597-4fb7-9892-92fa3538b881 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:52.375260 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-1c74f8f4-9f08-45d2-b2de-3eacc09cf5f5 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:52.377681 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1c74f8f4-9f08-45d2-b2de-3eacc09cf5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:52.379909 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be2298-3597-4fb7-9892-92fa3538b881 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:52.386978 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dde26a-04c1-45a0-a8bc-0eddb0dd0656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:52.387920 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dde26a-04c1-45a0-a8bc-0eddb0dd0656 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:52.388893 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dde26a-04c1-45a0-a8bc-0eddb0dd0656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:52.389269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dde26a-04c1-45a0-a8bc-0eddb0dd0656 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:52.390593 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dde26a-04c1-45a0-a8bc-0eddb0dd0656 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:52.392796 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dde26a-04c1-45a0-a8bc-0eddb0dd0656 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:52.426245 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:52.430679 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1069/2134] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:52.436370 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf05d2b-2040-448c-91d7-fbb2f4df6b6a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:52.437347 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf05d2b-2040-448c-91d7-fbb2f4df6b6a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:52.438514 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf05d2b-2040-448c-91d7-fbb2f4df6b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:52.453957 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1066/2135] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:52 2022] POST /identity/v3/projects => generated 373 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:52.455083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dde26a-04c1-45a0-a8bc-0eddb0dd0656 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:52.484636 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-4cf05d2b-2040-448c-91d7-fbb2f4df6b6a None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:52.488847 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4cf05d2b-2040-448c-91d7-fbb2f4df6b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:52.585695 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dde26a-04c1-45a0-a8bc-0eddb0dd0656 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:52.599126 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616bec4-adca-4504-a4fa-5a40c52220dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:52.600613 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616bec4-adca-4504-a4fa-5a40c52220dd admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:52.600901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616bec4-adca-4504-a4fa-5a40c52220dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:52.601224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616bec4-adca-4504-a4fa-5a40c52220dd admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:52.602118 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616bec4-adca-4504-a4fa-5a40c52220dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:52.604815 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616bec4-adca-4504-a4fa-5a40c52220dd admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:52.655953 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1067/2136] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:52 2022] POST /identity/v3/users => generated 287 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:52.661682 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1070/2137] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:52.663657 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616bec4-adca-4504-a4fa-5a40c52220dd admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:52.667920 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf05d2b-2040-448c-91d7-fbb2f4df6b6a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:52.697298 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616bec4-adca-4504-a4fa-5a40c52220dd admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:52.714777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf05d2b-2040-448c-91d7-fbb2f4df6b6a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:52.724382 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff0b13-a720-4b2e-bb45-8af6092e5d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:52.725176 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff0b13-a720-4b2e-bb45-8af6092e5d85 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:52.725762 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff0b13-a720-4b2e-bb45-8af6092e5d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:52.726115 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff0b13-a720-4b2e-bb45-8af6092e5d85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:52.727171 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff0b13-a720-4b2e-bb45-8af6092e5d85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:52.729919 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff0b13-a720-4b2e-bb45-8af6092e5d85 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:52.734511 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2c824-4b67-4f1a-8201-6f1bbc9ab0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:52.736227 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2c824-4b67-4f1a-8201-6f1bbc9ab0dd admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:52.737781 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2c824-4b67-4f1a-8201-6f1bbc9ab0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:52.737781 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2c824-4b67-4f1a-8201-6f1bbc9ab0dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:52.739466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c2c824-4b67-4f1a-8201-6f1bbc9ab0dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:52.740174 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c2c824-4b67-4f1a-8201-6f1bbc9ab0dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:52.743934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c2c824-4b67-4f1a-8201-6f1bbc9ab0dd admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:52.764429 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1068/2138] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:52 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:52.770672 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff0b13-a720-4b2e-bb45-8af6092e5d85 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:52.789011 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1071/2139] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:09:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:52.794098 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff0b13-a720-4b2e-bb45-8af6092e5d85 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:52.797272 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2c824-4b67-4f1a-8201-6f1bbc9ab0dd admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:52.809384 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19934f02-aae0-49a3-b9d3-1ab29244e171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:52.810434 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2c824-4b67-4f1a-8201-6f1bbc9ab0dd admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:52.811637 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19934f02-aae0-49a3-b9d3-1ab29244e171 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:52.812326 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19934f02-aae0-49a3-b9d3-1ab29244e171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:52.813044 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19934f02-aae0-49a3-b9d3-1ab29244e171 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:52.815083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19934f02-aae0-49a3-b9d3-1ab29244e171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:52.819989 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19934f02-aae0-49a3-b9d3-1ab29244e171 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:52.821284 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb74854-29dc-4b58-a029-33c48c29e268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:52.822127 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb74854-29dc-4b58-a029-33c48c29e268 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:52.822543 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb74854-29dc-4b58-a029-33c48c29e268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:52.822838 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb74854-29dc-4b58-a029-33c48c29e268 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:52.823869 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb74854-29dc-4b58-a029-33c48c29e268 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:52.826170 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb74854-29dc-4b58-a029-33c48c29e268 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:52.858501 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1069/2140] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:52 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:52.859609 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19934f02-aae0-49a3-b9d3-1ab29244e171 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:52.877445 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1072/2141] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:52 2022] POST /identity/v3/projects => generated 389 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:52.885532 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb74854-29dc-4b58-a029-33c48c29e268 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:52.898442 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19934f02-aae0-49a3-b9d3-1ab29244e171 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:53.032057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1494a8bd-a93e-488f-a1ca-9baa187a73fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:53.033135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494a8bd-a93e-488f-a1ca-9baa187a73fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:53.033705 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494a8bd-a93e-488f-a1ca-9baa187a73fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:53.034375 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494a8bd-a93e-488f-a1ca-9baa187a73fc admin admin] PATH_INFO: `/v3/projects/e834605e50024743b45a2600616de40b/users/1bd4928f1ad14fe4a757b184106b82e4/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:53.060407 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1494a8bd-a93e-488f-a1ca-9baa187a73fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e834605e50024743b45a2600616de40b, user_id=1bd4928f1ad14fe4a757b184106b82e4, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:53.063269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1494a8bd-a93e-488f-a1ca-9baa187a73fc admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:53.089955 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1070/2142] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:52 2022] PUT /identity/v3/projects/e834605e50024743b45a2600616de40b/users/1bd4928f1ad14fe4a757b184106b82e4/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:53.108367 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da4280-2029-4779-a987-d514a63f6888 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:53.108627 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da4280-2029-4779-a987-d514a63f6888 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:53.108793 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da4280-2029-4779-a987-d514a63f6888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:53.142027 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb74854-29dc-4b58-a029-33c48c29e268 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:53.165452 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-f9da4280-2029-4779-a987-d514a63f6888 None None] MFA Rules not processed for user `1bd4928f1ad14fe4a757b184106b82e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:53.168774 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-f9da4280-2029-4779-a987-d514a63f6888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:53.295602 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae3d04d-c133-402f-a7e7-b6ac288c651c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:53.296803 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3d04d-c133-402f-a7e7-b6ac288c651c admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:53.297607 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:53.298536 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3d04d-c133-402f-a7e7-b6ac288c651c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:53.299349 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3d04d-c133-402f-a7e7-b6ac288c651c admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:53.300722 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae3d04d-c133-402f-a7e7-b6ac288c651c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:53.310061 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae3d04d-c133-402f-a7e7-b6ac288c651c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:53.340298 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:53.403307 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1071/2143] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:53.407564 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1073/2144] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:52 2022] POST /identity/v3/users => generated 295 bytes in 524 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:09:53.421166 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da4280-2029-4779-a987-d514a63f6888 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:53.421855 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae3d04d-c133-402f-a7e7-b6ac288c651c admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:53.423128 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-f9da4280-2029-4779-a987-d514a63f6888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:53.454965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae3d04d-c133-402f-a7e7-b6ac288c651c admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:53.464082 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013105e3-5343-4819-b923-846a9599e81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:53.465224 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013105e3-5343-4819-b923-846a9599e81d admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:53.465712 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013105e3-5343-4819-b923-846a9599e81d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:53.466271 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013105e3-5343-4819-b923-846a9599e81d admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:53.467538 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013105e3-5343-4819-b923-846a9599e81d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:53.470798 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013105e3-5343-4819-b923-846a9599e81d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:53.519719 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1074/2145] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:53 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:53.521879 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013105e3-5343-4819-b923-846a9599e81d admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:53.574853 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013105e3-5343-4819-b923-846a9599e81d admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:53.597086 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe93f4e-892f-40f3-a55d-6a547d4fe19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:53.598238 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe93f4e-892f-40f3-a55d-6a547d4fe19b admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:53.601340 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe93f4e-892f-40f3-a55d-6a547d4fe19b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:53.601549 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe93f4e-892f-40f3-a55d-6a547d4fe19b admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:53.603629 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe93f4e-892f-40f3-a55d-6a547d4fe19b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:53.613132 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe93f4e-892f-40f3-a55d-6a547d4fe19b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:53.622716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da4280-2029-4779-a987-d514a63f6888 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:53.646338 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcbb9a2-aa0d-49fc-b258-8d8a9245bcfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:53.647471 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcbb9a2-aa0d-49fc-b258-8d8a9245bcfd service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:53.647964 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcbb9a2-aa0d-49fc-b258-8d8a9245bcfd service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:53.648486 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcbb9a2-aa0d-49fc-b258-8d8a9245bcfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:53.649665 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1075/2146] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:53 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:53.657269 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe93f4e-892f-40f3-a55d-6a547d4fe19b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:53.676177 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe93f4e-892f-40f3-a55d-6a547d4fe19b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:53.683361 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcbb9a2-aa0d-49fc-b258-8d8a9245bcfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:53.685198 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411740c-faba-40b8-a669-b2bc1ab6060b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:53.686620 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411740c-faba-40b8-a669-b2bc1ab6060b admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:53.686946 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcbb9a2-aa0d-49fc-b258-8d8a9245bcfd service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:53.687137 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411740c-faba-40b8-a669-b2bc1ab6060b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:53.687605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411740c-faba-40b8-a669-b2bc1ab6060b admin admin] PATH_INFO: `/v3/projects/c86a28aa09db413593b6047d4cbd0559/users/403896b926a1445b9895dc2d8d21854f/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:53.709045 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1072/2147] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:53.731185 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411740c-faba-40b8-a669-b2bc1ab6060b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c86a28aa09db413593b6047d4cbd0559, user_id=403896b926a1445b9895dc2d8d21854f, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:53.747576 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411740c-faba-40b8-a669-b2bc1ab6060b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:53.764985 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcbb9a2-aa0d-49fc-b258-8d8a9245bcfd service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:53.789110 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1076/2148] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:53 2022] PUT /identity/v3/projects/c86a28aa09db413593b6047d4cbd0559/users/403896b926a1445b9895dc2d8d21854f/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:53.798332 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1ecb9-80f9-4e3f-8ece-ddac72329acd None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:53.799184 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1ecb9-80f9-4e3f-8ece-ddac72329acd None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:53.806732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1ecb9-80f9-4e3f-8ece-ddac72329acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:53.810145 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcbb9a2-aa0d-49fc-b258-8d8a9245bcfd service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:53.875736 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-7da1ecb9-80f9-4e3f-8ece-ddac72329acd None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:53.877924 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7da1ecb9-80f9-4e3f-8ece-ddac72329acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:53.960772 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c0bbe-87bb-41be-90eb-f8f073554345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:53.961892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c0bbe-87bb-41be-90eb-f8f073554345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:53.962375 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c0bbe-87bb-41be-90eb-f8f073554345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:53.962871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c0bbe-87bb-41be-90eb-f8f073554345 admin admin] PATH_INFO: `/v3/users/1bd4928f1ad14fe4a757b184106b82e4` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:53.965314 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68c0bbe-87bb-41be-90eb-f8f073554345 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bd4928f1ad14fe4a757b184106b82e4)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:53.970210 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68c0bbe-87bb-41be-90eb-f8f073554345 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:54.014804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1077/2149] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:54.024616 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d7f2b-c04a-4515-b285-1fb7450f6228 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:54.025420 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d7f2b-c04a-4515-b285-1fb7450f6228 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:54.025995 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d7f2b-c04a-4515-b285-1fb7450f6228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:54.126564 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b68c0bbe-87bb-41be-90eb-f8f073554345 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bd4928f1ad14fe4a757b184106b82e4'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:54.127329 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b68c0bbe-87bb-41be-90eb-f8f073554345 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bd4928f1ad14fe4a757b184106b82e4'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:54.154957 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b68c0bbe-87bb-41be-90eb-f8f073554345 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bd4928f1ad14fe4a757b184106b82e4'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:54.157514 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-9e2d7f2b-c04a-4515-b285-1fb7450f6228 None None] MFA Rules not processed for user `ec40d75e772442d2a97e9bf18b4b90c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:54.161237 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9e2d7f2b-c04a-4515-b285-1fb7450f6228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:54.166697 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b68c0bbe-87bb-41be-90eb-f8f073554345 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bd4928f1ad14fe4a757b184106b82e4'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:54.193759 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-b68c0bbe-87bb-41be-90eb-f8f073554345 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bd4928f1ad14fe4a757b184106b82e4'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:54.204348 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1073/2150] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:53 2022] DELETE /identity/v3/users/1bd4928f1ad14fe4a757b184106b82e4 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:54.208968 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c0bbe-87bb-41be-90eb-f8f073554345 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:54.211943 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b68c0bbe-87bb-41be-90eb-f8f073554345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:54.223645 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:54.343486 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1078/2151] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:54.346948 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87845302-1415-42f7-ae19-369c0937f1d9 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:54.347924 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87845302-1415-42f7-ae19-369c0937f1d9 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:54.349149 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87845302-1415-42f7-ae19-369c0937f1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:54.387443 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-87845302-1415-42f7-ae19-369c0937f1d9 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:54.389000 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c0bbe-87bb-41be-90eb-f8f073554345 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:54.389710 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-87845302-1415-42f7-ae19-369c0937f1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:54.398588 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7a31f-5670-4ee1-b6e9-94932502d0ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:54.400873 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7a31f-5670-4ee1-b6e9-94932502d0ae service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:54.401701 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7a31f-5670-4ee1-b6e9-94932502d0ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:54.402557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7a31f-5670-4ee1-b6e9-94932502d0ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:54.410260 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-dec7a31f-5670-4ee1-b6e9-94932502d0ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:54.469903 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:54.554606 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1079/2152] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:54.560401 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87845302-1415-42f7-ae19-369c0937f1d9 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:54.561308 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-87845302-1415-42f7-ae19-369c0937f1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:54.582868 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87845302-1415-42f7-ae19-369c0937f1d9 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:54.599066 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e64ecb-b83b-42a8-bad6-60ec9e3b5720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:54.599747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e64ecb-b83b-42a8-bad6-60ec9e3b5720 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:54.601469 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e64ecb-b83b-42a8-bad6-60ec9e3b5720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:54.601469 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e64ecb-b83b-42a8-bad6-60ec9e3b5720 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:54.601817 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e64ecb-b83b-42a8-bad6-60ec9e3b5720 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:54.603856 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e64ecb-b83b-42a8-bad6-60ec9e3b5720 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:54.607182 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec7a31f-5670-4ee1-b6e9-94932502d0ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:54.608781 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec7a31f-5670-4ee1-b6e9-94932502d0ae service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:54.624660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1074/2153] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:54.625637 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7a31f-5670-4ee1-b6e9-94932502d0ae service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:54.652839 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7a31f-5670-4ee1-b6e9-94932502d0ae service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:54.665978 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d485fe-6ce0-4a2b-b3ce-6441159f0231 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:54.668191 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d485fe-6ce0-4a2b-b3ce-6441159f0231 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:54.669370 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d485fe-6ce0-4a2b-b3ce-6441159f0231 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:54.670104 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d485fe-6ce0-4a2b-b3ce-6441159f0231 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:54.675470 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1080/2154] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:54 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:54.682070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e64ecb-b83b-42a8-bad6-60ec9e3b5720 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:54.692726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d485fe-6ce0-4a2b-b3ce-6441159f0231 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:54.696605 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d485fe-6ce0-4a2b-b3ce-6441159f0231 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:54.722647 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1075/2155] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:54.808041 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e64ecb-b83b-42a8-bad6-60ec9e3b5720 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:54.817547 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a18bc1-a39d-4528-83b0-c3e3cb55dbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:54.818563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a18bc1-a39d-4528-83b0-c3e3cb55dbdd admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:54.819072 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a18bc1-a39d-4528-83b0-c3e3cb55dbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:54.820061 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a18bc1-a39d-4528-83b0-c3e3cb55dbdd admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:54.820886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a18bc1-a39d-4528-83b0-c3e3cb55dbdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:54.825370 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a18bc1-a39d-4528-83b0-c3e3cb55dbdd admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:54.867149 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1081/2156] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:54 2022] POST /identity/v3/users => generated 299 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:54.873799 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d485fe-6ce0-4a2b-b3ce-6441159f0231 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:54.910494 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d485fe-6ce0-4a2b-b3ce-6441159f0231 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:54.920812 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1ec89-f16e-42e3-90f5-d0c94c8d430a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:54.921829 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1ec89-f16e-42e3-90f5-d0c94c8d430a admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:54.922369 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1ec89-f16e-42e3-90f5-d0c94c8d430a admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:54.923073 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1ec89-f16e-42e3-90f5-d0c94c8d430a admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:54.924481 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d1ec89-f16e-42e3-90f5-d0c94c8d430a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:54.928112 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d1ec89-f16e-42e3-90f5-d0c94c8d430a admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:54.952389 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1076/2157] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:54.957988 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a18bc1-a39d-4528-83b0-c3e3cb55dbdd admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:54.977152 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a18bc1-a39d-4528-83b0-c3e3cb55dbdd admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:54.985564 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f521207-0f02-47f0-b4df-4ca6408f8d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:54.986316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f521207-0f02-47f0-b4df-4ca6408f8d1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:54.986585 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f521207-0f02-47f0-b4df-4ca6408f8d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:54.986868 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f521207-0f02-47f0-b4df-4ca6408f8d1a admin admin] PATH_INFO: `/v3/projects/da9c4987e8034e749a5d73796dfafa5b/users/ba2b258c27f44ba39e76eff0a8602a95/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:55.023485 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f521207-0f02-47f0-b4df-4ca6408f8d1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=da9c4987e8034e749a5d73796dfafa5b, user_id=ba2b258c27f44ba39e76eff0a8602a95, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:55.027074 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f521207-0f02-47f0-b4df-4ca6408f8d1a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:55.055810 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1082/2158] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:54 2022] PUT /identity/v3/projects/da9c4987e8034e749a5d73796dfafa5b/users/ba2b258c27f44ba39e76eff0a8602a95/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:55.061146 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1ec89-f16e-42e3-90f5-d0c94c8d430a admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:55.077786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1ec89-f16e-42e3-90f5-d0c94c8d430a admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:55.197105 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f5bc37-4351-4353-a590-9e6baf4aeccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:55.197908 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5bc37-4351-4353-a590-9e6baf4aeccc admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:55.198252 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5bc37-4351-4353-a590-9e6baf4aeccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:55.198597 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5bc37-4351-4353-a590-9e6baf4aeccc admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:55.199427 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f5bc37-4351-4353-a590-9e6baf4aeccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:55.201468 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f5bc37-4351-4353-a590-9e6baf4aeccc admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:55.238066 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f521207-0f02-47f0-b4df-4ca6408f8d1a admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:55.239891 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1077/2159] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:55 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:55.303156 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f521207-0f02-47f0-b4df-4ca6408f8d1a admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:55.311913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd8afe9-8624-470c-8572-5584cf3ccd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:55.313481 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8afe9-8624-470c-8572-5584cf3ccd1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:55.313956 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8afe9-8624-470c-8572-5584cf3ccd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:55.314635 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8afe9-8624-470c-8572-5584cf3ccd1c admin admin] PATH_INFO: `/v3/projects/da9c4987e8034e749a5d73796dfafa5b/users/ba2b258c27f44ba39e76eff0a8602a95/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:55.318755 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd8afe9-8624-470c-8572-5584cf3ccd1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=da9c4987e8034e749a5d73796dfafa5b, user_id=ba2b258c27f44ba39e76eff0a8602a95, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:55.324870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd8afe9-8624-470c-8572-5584cf3ccd1c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:55.347837 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1083/2160] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:55 2022] PUT /identity/v3/projects/da9c4987e8034e749a5d73796dfafa5b/users/ba2b258c27f44ba39e76eff0a8602a95/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:55.590507 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f5bc37-4351-4353-a590-9e6baf4aeccc admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:55.593069 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-22f5bc37-4351-4353-a590-9e6baf4aeccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:55.615649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f5bc37-4351-4353-a590-9e6baf4aeccc admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:55.701442 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:55.879533 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6111f84-aa47-416d-866c-bec1e39b6479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:55.881953 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6111f84-aa47-416d-866c-bec1e39b6479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:55.882462 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6111f84-aa47-416d-866c-bec1e39b6479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:55.885828 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6111f84-aa47-416d-866c-bec1e39b6479 admin admin] PATH_INFO: `/v3/projects/e834605e50024743b45a2600616de40b` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:55.891233 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6111f84-aa47-416d-866c-bec1e39b6479 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e834605e50024743b45a2600616de40b)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:55.899982 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6111f84-aa47-416d-866c-bec1e39b6479 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:56.156353 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d6111f84-aa47-416d-866c-bec1e39b6479 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e834605e50024743b45a2600616de40b', 'request_id': 'req-d6111f84-aa47-416d-866c-bec1e39b6479'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:56.178368 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d6111f84-aa47-416d-866c-bec1e39b6479 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e834605e50024743b45a2600616de40b', 'request_id': 'req-d6111f84-aa47-416d-866c-bec1e39b6479'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:56.225874 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d6111f84-aa47-416d-866c-bec1e39b6479 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e834605e50024743b45a2600616de40b', 'request_id': 'req-d6111f84-aa47-416d-866c-bec1e39b6479'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:56.282338 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d6111f84-aa47-416d-866c-bec1e39b6479 admin admin] The token cache is being invalidate because project e834605e50024743b45a2600616de40b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:56.283223 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d6111f84-aa47-416d-866c-bec1e39b6479 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:56.287675 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d6111f84-aa47-416d-866c-bec1e39b6479 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:56.298045 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1078/2161] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:55 2022] DELETE /identity/v3/projects/e834605e50024743b45a2600616de40b => generated 0 bytes in 708 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:56.306731 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a1168-ac5a-4a06-8105-abd23c111b9a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:56.307592 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a1168-ac5a-4a06-8105-abd23c111b9a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:56.307592 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a1168-ac5a-4a06-8105-abd23c111b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:56.390810 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c06a1168-ac5a-4a06-8105-abd23c111b9a None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:56.393257 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c06a1168-ac5a-4a06-8105-abd23c111b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:56.534777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1084/2162] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:56.548318 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454baa7-9af5-4337-948f-5f7d91d5e453 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:56.548783 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454baa7-9af5-4337-948f-5f7d91d5e453 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:56.549255 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454baa7-9af5-4337-948f-5f7d91d5e453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:56.644459 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-0454baa7-9af5-4337-948f-5f7d91d5e453 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:56.646988 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0454baa7-9af5-4337-948f-5f7d91d5e453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:56.705767 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1079/2163] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:56.712694 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a1168-ac5a-4a06-8105-abd23c111b9a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:56.724020 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:56.732947 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a1168-ac5a-4a06-8105-abd23c111b9a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:56.752424 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba4fc9-5efc-4095-8ebc-2aecf9f97212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:56.754109 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba4fc9-5efc-4095-8ebc-2aecf9f97212 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:56.754668 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba4fc9-5efc-4095-8ebc-2aecf9f97212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:56.755105 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba4fc9-5efc-4095-8ebc-2aecf9f97212 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:56.756259 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba4fc9-5efc-4095-8ebc-2aecf9f97212 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:56.757104 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba4fc9-5efc-4095-8ebc-2aecf9f97212 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:56.758910 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba4fc9-5efc-4095-8ebc-2aecf9f97212 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:56.813779 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1085/2164] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:09:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:56.816091 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0454baa7-9af5-4337-948f-5f7d91d5e453 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:56.864689 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0454baa7-9af5-4337-948f-5f7d91d5e453 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:56.879791 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13ce43-bf46-4cf8-a4ad-cfde816c72c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:56.880927 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13ce43-bf46-4cf8-a4ad-cfde816c72c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:56.881420 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13ce43-bf46-4cf8-a4ad-cfde816c72c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:56.881900 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13ce43-bf46-4cf8-a4ad-cfde816c72c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:56.883108 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad13ce43-bf46-4cf8-a4ad-cfde816c72c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:56.886965 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad13ce43-bf46-4cf8-a4ad-cfde816c72c5 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:57.014667 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1080/2165] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:56 2022] POST /identity/v3/projects => generated 397 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:57.019402 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba4fc9-5efc-4095-8ebc-2aecf9f97212 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:57.059406 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba4fc9-5efc-4095-8ebc-2aecf9f97212 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:57.452862 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003672ea-5239-4f18-be1e-9e54d10f3b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:57.453973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003672ea-5239-4f18-be1e-9e54d10f3b16 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:57.454480 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003672ea-5239-4f18-be1e-9e54d10f3b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:57.480426 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003672ea-5239-4f18-be1e-9e54d10f3b16 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:57.488361 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003672ea-5239-4f18-be1e-9e54d10f3b16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:57.510099 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003672ea-5239-4f18-be1e-9e54d10f3b16 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:57.668349 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13ce43-bf46-4cf8-a4ad-cfde816c72c5 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:57.671242 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ad13ce43-bf46-4cf8-a4ad-cfde816c72c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:57.714708 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13ce43-bf46-4cf8-a4ad-cfde816c72c5 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:57.751112 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59046706-b863-4d75-b72f-8f3ed57f8111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:57.756307 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59046706-b863-4d75-b72f-8f3ed57f8111 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:57.758870 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59046706-b863-4d75-b72f-8f3ed57f8111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:57.760689 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59046706-b863-4d75-b72f-8f3ed57f8111 admin admin] PATH_INFO: `/v3/users/55f61f8ca6d547569a1457baafe8aa27` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:57.780064 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1086/2166] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:57 2022] POST /identity/v3/users => generated 299 bytes in 757 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:57.780480 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59046706-b863-4d75-b72f-8f3ed57f8111 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55f61f8ca6d547569a1457baafe8aa27)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:57.783065 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003672ea-5239-4f18-be1e-9e54d10f3b16 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:57.797991 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59046706-b863-4d75-b72f-8f3ed57f8111 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:57.851168 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003672ea-5239-4f18-be1e-9e54d10f3b16 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:57.893427 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f3b4c-b713-4def-8514-27d8a146f573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:57.899112 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f3b4c-b713-4def-8514-27d8a146f573 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:57.899112 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f3b4c-b713-4def-8514-27d8a146f573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:57.913215 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f3b4c-b713-4def-8514-27d8a146f573 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:57.918990 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f3b4c-b713-4def-8514-27d8a146f573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:57.934178 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f3b4c-b713-4def-8514-27d8a146f573 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:58.045324 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1087/2167] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:58.051396 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f3b4c-b713-4def-8514-27d8a146f573 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:58.139533 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f3b4c-b713-4def-8514-27d8a146f573 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:58.157278 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-59046706-b863-4d75-b72f-8f3ed57f8111 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55f61f8ca6d547569a1457baafe8aa27'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:58.161725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-59046706-b863-4d75-b72f-8f3ed57f8111 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55f61f8ca6d547569a1457baafe8aa27'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:58.170282 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32feb48a-5559-4a5f-8bb6-0da18b59d253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:58.178065 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32feb48a-5559-4a5f-8bb6-0da18b59d253 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:58.195248 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32feb48a-5559-4a5f-8bb6-0da18b59d253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:58.195828 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32feb48a-5559-4a5f-8bb6-0da18b59d253 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:58.206953 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32feb48a-5559-4a5f-8bb6-0da18b59d253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:58.225118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32feb48a-5559-4a5f-8bb6-0da18b59d253 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:58.233996 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-59046706-b863-4d75-b72f-8f3ed57f8111 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55f61f8ca6d547569a1457baafe8aa27'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:58.278221 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-59046706-b863-4d75-b72f-8f3ed57f8111 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55f61f8ca6d547569a1457baafe8aa27'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:58.289209 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1088/2168] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:58.296183 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32feb48a-5559-4a5f-8bb6-0da18b59d253 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:58.302579 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-32feb48a-5559-4a5f-8bb6-0da18b59d253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:58.327969 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-59046706-b863-4d75-b72f-8f3ed57f8111 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55f61f8ca6d547569a1457baafe8aa27'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:58.331613 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:58.368878 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1081/2169] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:57 2022] DELETE /identity/v3/users/55f61f8ca6d547569a1457baafe8aa27 => generated 0 bytes in 695 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:58.374211 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dfa38d-8c39-435e-9914-7e73ebe2efd8 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:58.374671 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dfa38d-8c39-435e-9914-7e73ebe2efd8 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:58.375161 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dfa38d-8c39-435e-9914-7e73ebe2efd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:58.387074 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32feb48a-5559-4a5f-8bb6-0da18b59d253 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:58.465671 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-27dfa38d-8c39-435e-9914-7e73ebe2efd8 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:58.467901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-27dfa38d-8c39-435e-9914-7e73ebe2efd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:58.576210 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc44596-f79b-4260-a499-6cb4423cadcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:58.578543 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc44596-f79b-4260-a499-6cb4423cadcb admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:58.579918 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc44596-f79b-4260-a499-6cb4423cadcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:58.582381 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc44596-f79b-4260-a499-6cb4423cadcb admin admin] PATH_INFO: `/v3/projects/fe63d5f4cc644128912d157da6ea8e41/users/aae0fa8c628e47a4b207ab4f281f271d/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:58.628193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc44596-f79b-4260-a499-6cb4423cadcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe63d5f4cc644128912d157da6ea8e41, user_id=aae0fa8c628e47a4b207ab4f281f271d, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:58.637065 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc44596-f79b-4260-a499-6cb4423cadcb admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:58.688507 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1089/2170] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:58 2022] PUT /identity/v3/projects/fe63d5f4cc644128912d157da6ea8e41/users/aae0fa8c628e47a4b207ab4f281f271d/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:58.699887 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06c9cb-c972-4e69-a244-1486f051610a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:58.700919 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06c9cb-c972-4e69-a244-1486f051610a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:58.702688 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06c9cb-c972-4e69-a244-1486f051610a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:58.705430 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1082/2171] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:58.782246 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-1c06c9cb-c972-4e69-a244-1486f051610a None None] MFA Rules not processed for user `aae0fa8c628e47a4b207ab4f281f271d`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:58.783448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1c06c9cb-c972-4e69-a244-1486f051610a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:58.807934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dfa38d-8c39-435e-9914-7e73ebe2efd8 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:58.810805 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-27dfa38d-8c39-435e-9914-7e73ebe2efd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:58.866488 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:58.977914 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1090/2172] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:09:58 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:58.997506 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06c9cb-c972-4e69-a244-1486f051610a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:58.999651 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1c06c9cb-c972-4e69-a244-1486f051610a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:59.081219 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dfa38d-8c39-435e-9914-7e73ebe2efd8 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:59.172802 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06c9cb-c972-4e69-a244-1486f051610a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:59.181789 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bf778d-e68b-4eb8-9f98-270a91d1233c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:59.183503 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf778d-e68b-4eb8-9f98-270a91d1233c service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:59.184098 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf778d-e68b-4eb8-9f98-270a91d1233c service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:59.184848 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf778d-e68b-4eb8-9f98-270a91d1233c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:59.204481 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bf778d-e68b-4eb8-9f98-270a91d1233c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:59.207704 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bf778d-e68b-4eb8-9f98-270a91d1233c service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:59.235677 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1091/2173] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:58 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:09:59.264491 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373efa76-1017-4fae-b648-1ee448e68c15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:59.266402 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373efa76-1017-4fae-b648-1ee448e68c15 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:59.266402 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373efa76-1017-4fae-b648-1ee448e68c15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:59.266846 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373efa76-1017-4fae-b648-1ee448e68c15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:59.444992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373efa76-1017-4fae-b648-1ee448e68c15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:59.448956 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373efa76-1017-4fae-b648-1ee448e68c15 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:59.473459 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1083/2174] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 666 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:10:01.069469 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bf778d-e68b-4eb8-9f98-270a91d1233c service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:01.071222 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-15bf778d-e68b-4eb8-9f98-270a91d1233c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:01.102259 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bf778d-e68b-4eb8-9f98-270a91d1233c service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:01.122203 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c255d68-532b-470f-afad-06c4d694f1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:01.128579 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c255d68-532b-470f-afad-06c4d694f1b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:01.129093 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c255d68-532b-470f-afad-06c4d694f1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:01.129587 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c255d68-532b-470f-afad-06c4d694f1b7 admin admin] PATH_INFO: `/v3/users/099d463ec7bc43a7a868e785cfdf659b` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:01.137875 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c255d68-532b-470f-afad-06c4d694f1b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=099d463ec7bc43a7a868e785cfdf659b)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:01.147370 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c255d68-532b-470f-afad-06c4d694f1b7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:01.370508 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4c255d68-532b-470f-afad-06c4d694f1b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '099d463ec7bc43a7a868e785cfdf659b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:01.372352 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4c255d68-532b-470f-afad-06c4d694f1b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '099d463ec7bc43a7a868e785cfdf659b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:01.466497 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4c255d68-532b-470f-afad-06c4d694f1b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '099d463ec7bc43a7a868e785cfdf659b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:01.506104 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4c255d68-532b-470f-afad-06c4d694f1b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '099d463ec7bc43a7a868e785cfdf659b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:01.517194 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4c255d68-532b-470f-afad-06c4d694f1b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '099d463ec7bc43a7a868e785cfdf659b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:01.586623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1092/2175] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:10:01 2022] DELETE /identity/v3/users/099d463ec7bc43a7a868e785cfdf659b => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:01.593702 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ccf23-9388-4730-a583-ca74c6313134 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:01.594469 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ccf23-9388-4730-a583-ca74c6313134 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:01.594697 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ccf23-9388-4730-a583-ca74c6313134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:01.627419 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c255d68-532b-470f-afad-06c4d694f1b7 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:01.630780 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4c255d68-532b-470f-afad-06c4d694f1b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:01.637298 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:01.697167 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-1f6ccf23-9388-4730-a583-ca74c6313134 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:01.698843 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1f6ccf23-9388-4730-a583-ca74c6313134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:01.836486 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:01.875715 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c255d68-532b-470f-afad-06c4d694f1b7 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:01.891845 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febfd530-c3a2-4f3d-9940-9e431d95f8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:01.892662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfd530-c3a2-4f3d-9940-9e431d95f8c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:01.893062 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfd530-c3a2-4f3d-9940-9e431d95f8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:01.894293 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfd530-c3a2-4f3d-9940-9e431d95f8c4 admin admin] PATH_INFO: `/v3/projects/cf0e090327da468196d349b6213848e9` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:01.894997 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febfd530-c3a2-4f3d-9940-9e431d95f8c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf0e090327da468196d349b6213848e9)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:01.899771 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febfd530-c3a2-4f3d-9940-9e431d95f8c4 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:01.939990 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1084/2176] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:01.978574 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ccf23-9388-4730-a583-ca74c6313134 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:01.981870 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1f6ccf23-9388-4730-a583-ca74c6313134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:02.109479 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-febfd530-c3a2-4f3d-9940-9e431d95f8c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf0e090327da468196d349b6213848e9', 'request_id': 'req-febfd530-c3a2-4f3d-9940-9e431d95f8c4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:02.146404 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-febfd530-c3a2-4f3d-9940-9e431d95f8c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf0e090327da468196d349b6213848e9', 'request_id': 'req-febfd530-c3a2-4f3d-9940-9e431d95f8c4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:02.177690 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-febfd530-c3a2-4f3d-9940-9e431d95f8c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf0e090327da468196d349b6213848e9', 'request_id': 'req-febfd530-c3a2-4f3d-9940-9e431d95f8c4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:02.192544 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ccf23-9388-4730-a583-ca74c6313134 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:02.225571 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-febfd530-c3a2-4f3d-9940-9e431d95f8c4 admin admin] The token cache is being invalidate because project cf0e090327da468196d349b6213848e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:10:02.225571 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-febfd530-c3a2-4f3d-9940-9e431d95f8c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:02.232144 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-febfd530-c3a2-4f3d-9940-9e431d95f8c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:02.234556 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1093/2177] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:10:01 2022] DELETE /identity/v3/projects/cf0e090327da468196d349b6213848e9 => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:02.244053 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24216205-414b-4dfe-9c7b-1bdb4aefc65e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:02.244643 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24216205-414b-4dfe-9c7b-1bdb4aefc65e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:02.245114 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24216205-414b-4dfe-9c7b-1bdb4aefc65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:02.302476 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-24216205-414b-4dfe-9c7b-1bdb4aefc65e None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:02.313438 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-24216205-414b-4dfe-9c7b-1bdb4aefc65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:02.449341 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f519083-cd45-42a2-80e4-06b3f20b3b6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:02.461812 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f519083-cd45-42a2-80e4-06b3f20b3b6e service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:02.462643 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f519083-cd45-42a2-80e4-06b3f20b3b6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:02.463792 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f519083-cd45-42a2-80e4-06b3f20b3b6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:02.468503 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3f519083-cd45-42a2-80e4-06b3f20b3b6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:02.708314 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1094/2178] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:02.715819 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026cdf2-a1f0-4b56-bcb0-66db4fdb0da3 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:02.716684 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026cdf2-a1f0-4b56-bcb0-66db4fdb0da3 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:02.717507 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026cdf2-a1f0-4b56-bcb0-66db4fdb0da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:02.773266 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f519083-cd45-42a2-80e4-06b3f20b3b6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:02.777557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f519083-cd45-42a2-80e4-06b3f20b3b6e service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:02.810311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1085/2179] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 829 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:10:02.821461 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-0026cdf2-a1f0-4b56-bcb0-66db4fdb0da3 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:02.822982 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0026cdf2-a1f0-4b56-bcb0-66db4fdb0da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:02.852795 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1095/2180] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:02.854737 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f519083-cd45-42a2-80e4-06b3f20b3b6e service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:02.872490 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f519083-cd45-42a2-80e4-06b3f20b3b6e service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:02.897995 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378248e2-ad3b-418d-8b0a-8081c81dbf80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:02.898839 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378248e2-ad3b-418d-8b0a-8081c81dbf80 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:02.899233 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378248e2-ad3b-418d-8b0a-8081c81dbf80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:02.900314 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378248e2-ad3b-418d-8b0a-8081c81dbf80 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:02.901784 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378248e2-ad3b-418d-8b0a-8081c81dbf80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:10:02.901784 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378248e2-ad3b-418d-8b0a-8081c81dbf80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:02.919640 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378248e2-ad3b-418d-8b0a-8081c81dbf80 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:02.936077 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:02.963650 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026cdf2-a1f0-4b56-bcb0-66db4fdb0da3 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:02.966677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1086/2181] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:10:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:10:02.983327 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026cdf2-a1f0-4b56-bcb0-66db4fdb0da3 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:03.000332 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d65ba-df8b-4702-9d25-309362ae7a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:03.001428 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d65ba-df8b-4702-9d25-309362ae7a72 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:03.001932 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d65ba-df8b-4702-9d25-309362ae7a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:03.002431 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d65ba-df8b-4702-9d25-309362ae7a72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:03.003794 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43d65ba-df8b-4702-9d25-309362ae7a72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:03.028033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43d65ba-df8b-4702-9d25-309362ae7a72 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:03.166872 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1096/2182] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:10:02 2022] POST /identity/v3/projects => generated 369 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:10:03.168618 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378248e2-ad3b-418d-8b0a-8081c81dbf80 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:03.250275 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378248e2-ad3b-418d-8b0a-8081c81dbf80 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:03.511080 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92405d8-9bc9-4108-ac96-615bb6f4ad78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:03.511858 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92405d8-9bc9-4108-ac96-615bb6f4ad78 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:03.512216 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92405d8-9bc9-4108-ac96-615bb6f4ad78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:03.512619 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92405d8-9bc9-4108-ac96-615bb6f4ad78 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:03.513411 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92405d8-9bc9-4108-ac96-615bb6f4ad78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:03.516223 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92405d8-9bc9-4108-ac96-615bb6f4ad78 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:03.581529 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1087/2183] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:10:03 2022] POST /identity/v3/users => generated 285 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:10:03.587970 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d65ba-df8b-4702-9d25-309362ae7a72 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:03.634352 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d65ba-df8b-4702-9d25-309362ae7a72 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:03.675405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba882fd6-0d72-482d-9749-898e8193994c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:03.677435 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba882fd6-0d72-482d-9749-898e8193994c admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:03.678264 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba882fd6-0d72-482d-9749-898e8193994c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:03.679061 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba882fd6-0d72-482d-9749-898e8193994c admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:03.681258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba882fd6-0d72-482d-9749-898e8193994c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:03.690679 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba882fd6-0d72-482d-9749-898e8193994c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:03.730071 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1097/2184] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:10:03.731408 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92405d8-9bc9-4108-ac96-615bb6f4ad78 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:03.777784 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92405d8-9bc9-4108-ac96-615bb6f4ad78 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:03.802803 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c4ffe-b5bc-45b4-afe0-54a0ce1499ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:03.803921 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c4ffe-b5bc-45b4-afe0-54a0ce1499ac admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:03.804526 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c4ffe-b5bc-45b4-afe0-54a0ce1499ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:03.806165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c4ffe-b5bc-45b4-afe0-54a0ce1499ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:03.806350 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c4ffe-b5bc-45b4-afe0-54a0ce1499ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:03.819358 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c4ffe-b5bc-45b4-afe0-54a0ce1499ac admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:03.868300 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1088/2185] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:10:03.877388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba882fd6-0d72-482d-9749-898e8193994c admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:03.911804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba882fd6-0d72-482d-9749-898e8193994c admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:03.936213 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2602198-f028-4235-bd67-f2b4d873e14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:03.940890 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2602198-f028-4235-bd67-f2b4d873e14d admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:03.942506 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2602198-f028-4235-bd67-f2b4d873e14d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:03.942993 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2602198-f028-4235-bd67-f2b4d873e14d admin admin] PATH_INFO: `/v3/projects/fb3f97da433d4d83b8c2837575ab11f1/users/58140e8aa8e9496bb41f962dab03227a/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:03.963423 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265c609-b68d-49cb-9ec9-94ac97b9625c None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:03.967285 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265c609-b68d-49cb-9ec9-94ac97b9625c None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:03.968194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265c609-b68d-49cb-9ec9-94ac97b9625c None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:03.976409 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1089/2186] 10.222.0.45 () {54 vars in 921 bytes} [Mon Aug 1 14:10:03 2022] GET /identity => generated 270 bytes in 28 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:10:04.013193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ae44e-af37-459c-a5d4-114d46a018df None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:04.017879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ae44e-af37-459c-a5d4-114d46a018df None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:04.020532 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ae44e-af37-459c-a5d4-114d46a018df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:04.030916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2602198-f028-4235-bd67-f2b4d873e14d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb3f97da433d4d83b8c2837575ab11f1, user_id=58140e8aa8e9496bb41f962dab03227a, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:04.047840 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2602198-f028-4235-bd67-f2b4d873e14d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:04.072002 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1098/2187] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:10:03 2022] PUT /identity/v3/projects/fb3f97da433d4d83b8c2837575ab11f1/users/58140e8aa8e9496bb41f962dab03227a/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:04.074985 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2602198-f028-4235-bd67-f2b4d873e14d admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:04.082118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a2602198-f028-4235-bd67-f2b4d873e14d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:04.116625 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-b51ae44e-af37-459c-a5d4-114d46a018df None None] MFA Rules not processed for user `58380e30789041e4ad9e8652bc628658`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:04.124237 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b51ae44e-af37-459c-a5d4-114d46a018df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:04.140839 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:04.406044 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1090/2188] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Aug 1 14:10:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:04.415816 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7020b-d2e6-4de2-960b-17a86d6c8896 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:04.416330 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7020b-d2e6-4de2-960b-17a86d6c8896 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:04.416822 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7020b-d2e6-4de2-960b-17a86d6c8896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:04.462795 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2602198-f028-4235-bd67-f2b4d873e14d admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:04.478084 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84953a-51af-4b86-8a08-5f55eabd8b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:04.478766 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84953a-51af-4b86-8a08-5f55eabd8b0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:04.479244 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84953a-51af-4b86-8a08-5f55eabd8b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:04.479513 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84953a-51af-4b86-8a08-5f55eabd8b0b admin admin] PATH_INFO: `/v3/users/e317f73780184baa87733294b8db5fe7` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:04.481460 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d84953a-51af-4b86-8a08-5f55eabd8b0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e317f73780184baa87733294b8db5fe7)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:04.483707 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d84953a-51af-4b86-8a08-5f55eabd8b0b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:04.516740 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-3fb7020b-d2e6-4de2-960b-17a86d6c8896 None None] MFA Rules not processed for user `58140e8aa8e9496bb41f962dab03227a`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:04.525372 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3fb7020b-d2e6-4de2-960b-17a86d6c8896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:04.693237 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7d84953a-51af-4b86-8a08-5f55eabd8b0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e317f73780184baa87733294b8db5fe7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:04.694442 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7d84953a-51af-4b86-8a08-5f55eabd8b0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e317f73780184baa87733294b8db5fe7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:04.773040 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1091/2189] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:04 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:04.773040 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7020b-d2e6-4de2-960b-17a86d6c8896 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:04.776103 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7d84953a-51af-4b86-8a08-5f55eabd8b0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e317f73780184baa87733294b8db5fe7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:04.787304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3fb7020b-d2e6-4de2-960b-17a86d6c8896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:04.803586 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7d84953a-51af-4b86-8a08-5f55eabd8b0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e317f73780184baa87733294b8db5fe7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:04.804827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7d84953a-51af-4b86-8a08-5f55eabd8b0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e317f73780184baa87733294b8db5fe7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:04.835276 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:04.855886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1099/2190] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:10:04 2022] DELETE /identity/v3/users/e317f73780184baa87733294b8db5fe7 => generated 0 bytes in 781 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:10:04.857931 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84953a-51af-4b86-8a08-5f55eabd8b0b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:04.858206 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7020b-d2e6-4de2-960b-17a86d6c8896 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:04.866487 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7d84953a-51af-4b86-8a08-5f55eabd8b0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:04.993793 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84953a-51af-4b86-8a08-5f55eabd8b0b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:04.998456 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220618e4-a95c-419b-a7f4-056e51dd26c9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7317c8829cb4fa1b30711f1db575cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:04.999144 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220618e4-a95c-419b-a7f4-056e51dd26c9 service glance] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:04.999488 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220618e4-a95c-419b-a7f4-056e51dd26c9 service glance] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:04.999725 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220618e4-a95c-419b-a7f4-056e51dd26c9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:05.002099 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-220618e4-a95c-419b-a7f4-056e51dd26c9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:05.003790 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fbb0d1-c3c1-4eed-8e35-4e09b8568045 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58380e30789041e4ad9e8652bc628658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:05.004703 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbb0d1-c3c1-4eed-8e35-4e09b8568045 service placement] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:05.005843 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbb0d1-c3c1-4eed-8e35-4e09b8568045 service placement] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:05.006127 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbb0d1-c3c1-4eed-8e35-4e09b8568045 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:05.008193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-51fbb0d1-c3c1-4eed-8e35-4e09b8568045 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:05.025282 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fbb0d1-c3c1-4eed-8e35-4e09b8568045 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:05.027071 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fbb0d1-c3c1-4eed-8e35-4e09b8568045 service placement] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:05.039982 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1092/2191] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:10:05.040952 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fbb0d1-c3c1-4eed-8e35-4e09b8568045 service placement] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:05.042502 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-51fbb0d1-c3c1-4eed-8e35-4e09b8568045 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:05.140512 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:05.176504 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220618e4-a95c-419b-a7f4-056e51dd26c9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:05.179653 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220618e4-a95c-419b-a7f4-056e51dd26c9 service glance] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:05.237027 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1100/2192] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:04 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:10:05.247417 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fbb0d1-c3c1-4eed-8e35-4e09b8568045 service placement] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:05.283434 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11730333-ac39-4138-8d9f-6aba9a75b8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:05.285314 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11730333-ac39-4138-8d9f-6aba9a75b8ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:05.294940 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11730333-ac39-4138-8d9f-6aba9a75b8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:05.295440 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11730333-ac39-4138-8d9f-6aba9a75b8ef admin admin] PATH_INFO: `/v3/projects/6a2e679e5e5e415284ead8a579ad3dd3` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:05.304367 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11730333-ac39-4138-8d9f-6aba9a75b8ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a2e679e5e5e415284ead8a579ad3dd3)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:05.319573 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11730333-ac39-4138-8d9f-6aba9a75b8ef admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:05.459238 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-11730333-ac39-4138-8d9f-6aba9a75b8ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a2e679e5e5e415284ead8a579ad3dd3', 'request_id': 'req-11730333-ac39-4138-8d9f-6aba9a75b8ef'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:05.471195 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-11730333-ac39-4138-8d9f-6aba9a75b8ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a2e679e5e5e415284ead8a579ad3dd3', 'request_id': 'req-11730333-ac39-4138-8d9f-6aba9a75b8ef'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:05.488347 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-11730333-ac39-4138-8d9f-6aba9a75b8ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a2e679e5e5e415284ead8a579ad3dd3', 'request_id': 'req-11730333-ac39-4138-8d9f-6aba9a75b8ef'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:05.527732 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-11730333-ac39-4138-8d9f-6aba9a75b8ef admin admin] The token cache is being invalidate because project 6a2e679e5e5e415284ead8a579ad3dd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:10:05.528359 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-11730333-ac39-4138-8d9f-6aba9a75b8ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:05.529891 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-11730333-ac39-4138-8d9f-6aba9a75b8ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:05.532811 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1093/2193] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:10:05 2022] DELETE /identity/v3/projects/6a2e679e5e5e415284ead8a579ad3dd3 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:05.540077 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb75f9-2a98-455a-94a9-de79d97a48c3 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:05.541323 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb75f9-2a98-455a-94a9-de79d97a48c3 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:05.541913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb75f9-2a98-455a-94a9-de79d97a48c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:05.604881 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d7bb75f9-2a98-455a-94a9-de79d97a48c3 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:05.608839 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d7bb75f9-2a98-455a-94a9-de79d97a48c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:05.763885 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1101/2194] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:05.775672 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5d9af-29a4-4aa5-94cf-4aa75327532a None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:05.777810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5d9af-29a4-4aa5-94cf-4aa75327532a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:05.778681 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5d9af-29a4-4aa5-94cf-4aa75327532a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:05.910032 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-eef5d9af-29a4-4aa5-94cf-4aa75327532a None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:05.924553 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-eef5d9af-29a4-4aa5-94cf-4aa75327532a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:05.979930 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1094/2195] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:10:05.988181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bb75f9-2a98-455a-94a9-de79d97a48c3 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:06.016590 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bb75f9-2a98-455a-94a9-de79d97a48c3 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:06.034965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2362dfb-1db7-424a-99dd-e50057c3a3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:06.037859 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2362dfb-1db7-424a-99dd-e50057c3a3c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:06.038700 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2362dfb-1db7-424a-99dd-e50057c3a3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:06.039427 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2362dfb-1db7-424a-99dd-e50057c3a3c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:06.041144 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2362dfb-1db7-424a-99dd-e50057c3a3c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:10:06.043766 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2362dfb-1db7-424a-99dd-e50057c3a3c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:06.060582 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2362dfb-1db7-424a-99dd-e50057c3a3c5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:06.077057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5d9af-29a4-4aa5-94cf-4aa75327532a None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:06.080102 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-eef5d9af-29a4-4aa5-94cf-4aa75327532a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:06.130349 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5d9af-29a4-4aa5-94cf-4aa75327532a None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:06.132070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1102/2196] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:10:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:10:06.135066 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2362dfb-1db7-424a-99dd-e50057c3a3c5 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:06.182176 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc44204-e5d2-4891-b0c2-86f130af6d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:06.183407 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc44204-e5d2-4891-b0c2-86f130af6d68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:06.186036 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc44204-e5d2-4891-b0c2-86f130af6d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:06.189287 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc44204-e5d2-4891-b0c2-86f130af6d68 admin admin] PATH_INFO: `/v3/projects/5166861cf436411e9797c575c32bb800` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:06.197246 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc44204-e5d2-4891-b0c2-86f130af6d68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5166861cf436411e9797c575c32bb800)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:06.207311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc44204-e5d2-4891-b0c2-86f130af6d68 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:06.208963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2362dfb-1db7-424a-99dd-e50057c3a3c5 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:06.238809 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfaad12-3140-435d-b7b2-0d4970ca19bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:06.240081 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfaad12-3140-435d-b7b2-0d4970ca19bc admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:06.240724 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfaad12-3140-435d-b7b2-0d4970ca19bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:06.241525 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfaad12-3140-435d-b7b2-0d4970ca19bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:06.247380 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfaad12-3140-435d-b7b2-0d4970ca19bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:06.261424 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfaad12-3140-435d-b7b2-0d4970ca19bc admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:06.310301 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:06.477567 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1103/2197] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:10:06 2022] POST /identity/v3/projects => generated 373 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:10:06.483814 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfaad12-3140-435d-b7b2-0d4970ca19bc admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:06.522314 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1bc44204-e5d2-4891-b0c2-86f130af6d68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5166861cf436411e9797c575c32bb800', 'request_id': 'req-1bc44204-e5d2-4891-b0c2-86f130af6d68'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:06.538269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1bc44204-e5d2-4891-b0c2-86f130af6d68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5166861cf436411e9797c575c32bb800', 'request_id': 'req-1bc44204-e5d2-4891-b0c2-86f130af6d68'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:06.601730 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1bc44204-e5d2-4891-b0c2-86f130af6d68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5166861cf436411e9797c575c32bb800', 'request_id': 'req-1bc44204-e5d2-4891-b0c2-86f130af6d68'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:06.651767 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1bc44204-e5d2-4891-b0c2-86f130af6d68 admin admin] The token cache is being invalidate because project 5166861cf436411e9797c575c32bb800 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:10:06.652109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1bc44204-e5d2-4891-b0c2-86f130af6d68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:06.655320 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1bc44204-e5d2-4891-b0c2-86f130af6d68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:06.661414 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1095/2198] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:10:06 2022] DELETE /identity/v3/projects/5166861cf436411e9797c575c32bb800 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:06.671213 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc44204-e5d2-4891-b0c2-86f130af6d68 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:06.673866 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1bc44204-e5d2-4891-b0c2-86f130af6d68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:06.814768 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfaad12-3140-435d-b7b2-0d4970ca19bc admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:06.830426 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88ff03-6086-47d9-8833-2b6adee43e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:06.831653 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88ff03-6086-47d9-8833-2b6adee43e4e admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:06.832162 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88ff03-6086-47d9-8833-2b6adee43e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:06.832699 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88ff03-6086-47d9-8833-2b6adee43e4e admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:06.843296 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88ff03-6086-47d9-8833-2b6adee43e4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:06.851423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88ff03-6086-47d9-8833-2b6adee43e4e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:06.949886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1104/2199] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:10:06 2022] POST /identity/v3/users => generated 287 bytes in 467 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:10:06.954466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88ff03-6086-47d9-8833-2b6adee43e4e admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:06.957030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-eb88ff03-6086-47d9-8833-2b6adee43e4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:07.003271 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88ff03-6086-47d9-8833-2b6adee43e4e admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:07.023779 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc44204-e5d2-4891-b0c2-86f130af6d68 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:07.026513 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06302c-651e-49eb-b471-14cc0ca796f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:07.026513 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06302c-651e-49eb-b471-14cc0ca796f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:07.028186 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06302c-651e-49eb-b471-14cc0ca796f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:07.029397 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06302c-651e-49eb-b471-14cc0ca796f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:07.035521 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d06302c-651e-49eb-b471-14cc0ca796f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:07.042308 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf3e26-8616-43c9-b292-7eb8ac05f471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:07.044480 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf3e26-8616-43c9-b292-7eb8ac05f471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:07.044991 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf3e26-8616-43c9-b292-7eb8ac05f471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:07.045480 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d06302c-651e-49eb-b471-14cc0ca796f3 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:07.045670 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf3e26-8616-43c9-b292-7eb8ac05f471 admin admin] PATH_INFO: `/v3/users/c44dad19055447d9a24206c441e2a0ab` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:07.051193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebf3e26-8616-43c9-b292-7eb8ac05f471 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c44dad19055447d9a24206c441e2a0ab)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:07.055324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebf3e26-8616-43c9-b292-7eb8ac05f471 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:07.098768 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1105/2200] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:10:07.106235 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06302c-651e-49eb-b471-14cc0ca796f3 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:07.129936 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06302c-651e-49eb-b471-14cc0ca796f3 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:07.150855 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be353b-9072-410b-9c8c-3825637f2605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:07.153257 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be353b-9072-410b-9c8c-3825637f2605 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:07.158046 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be353b-9072-410b-9c8c-3825637f2605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:07.159077 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be353b-9072-410b-9c8c-3825637f2605 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:07.160898 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be353b-9072-410b-9c8c-3825637f2605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:07.170566 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be353b-9072-410b-9c8c-3825637f2605 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:07.210436 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1106/2201] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:07 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:10:07.216879 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be353b-9072-410b-9c8c-3825637f2605 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:07.255331 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be353b-9072-410b-9c8c-3825637f2605 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:07.276025 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aeb276-84a2-4b12-8756-3eb21be89c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:07.282280 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aeb276-84a2-4b12-8756-3eb21be89c5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:07.283482 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aeb276-84a2-4b12-8756-3eb21be89c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:07.283669 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1ebf3e26-8616-43c9-b292-7eb8ac05f471 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c44dad19055447d9a24206c441e2a0ab'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:07.284785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aeb276-84a2-4b12-8756-3eb21be89c5b admin admin] PATH_INFO: `/v3/projects/3eba5c871a224532bb7a1f48891d3f50/users/e0ce1e490e5741bdbe1cee1c0426ae1e/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:07.286370 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1ebf3e26-8616-43c9-b292-7eb8ac05f471 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c44dad19055447d9a24206c441e2a0ab'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:07.319910 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1ebf3e26-8616-43c9-b292-7eb8ac05f471 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c44dad19055447d9a24206c441e2a0ab'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:07.337369 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1ebf3e26-8616-43c9-b292-7eb8ac05f471 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c44dad19055447d9a24206c441e2a0ab'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:07.339551 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aeb276-84a2-4b12-8756-3eb21be89c5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eba5c871a224532bb7a1f48891d3f50, user_id=e0ce1e490e5741bdbe1cee1c0426ae1e, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:07.350019 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aeb276-84a2-4b12-8756-3eb21be89c5b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:07.369609 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:07.395743 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1ebf3e26-8616-43c9-b292-7eb8ac05f471 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c44dad19055447d9a24206c441e2a0ab'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:07.443102 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1107/2202] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:10:07 2022] PUT /identity/v3/projects/3eba5c871a224532bb7a1f48891d3f50/users/e0ce1e490e5741bdbe1cee1c0426ae1e/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:07.457268 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb978c-1a92-4f57-9e34-f39bb0c577ab None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:07.457774 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb978c-1a92-4f57-9e34-f39bb0c577ab None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:07.458457 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb978c-1a92-4f57-9e34-f39bb0c577ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:07.462199 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1096/2203] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:10:06 2022] DELETE /identity/v3/users/c44dad19055447d9a24206c441e2a0ab => generated 0 bytes in 780 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:07.573262 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b0bb978c-1a92-4f57-9e34-f39bb0c577ab None None] MFA Rules not processed for user `e0ce1e490e5741bdbe1cee1c0426ae1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:07.582970 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b0bb978c-1a92-4f57-9e34-f39bb0c577ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:07.865122 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1108/2204] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:07.874718 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf3e26-8616-43c9-b292-7eb8ac05f471 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:07.878139 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1ebf3e26-8616-43c9-b292-7eb8ac05f471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:08.001061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:08.433512 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf3e26-8616-43c9-b292-7eb8ac05f471 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:08.471952 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19280640-696f-46c8-ad30-bd2e36048419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:08.479700 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19280640-696f-46c8-ad30-bd2e36048419 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:08.480220 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19280640-696f-46c8-ad30-bd2e36048419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:08.480831 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19280640-696f-46c8-ad30-bd2e36048419 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:08.482264 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19280640-696f-46c8-ad30-bd2e36048419 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:08.498356 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19280640-696f-46c8-ad30-bd2e36048419 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:08.658565 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb978c-1a92-4f57-9e34-f39bb0c577ab None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:08.662938 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1097/2205] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:10:07 2022] POST /identity/v3/projects => generated 373 bytes in 791 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:10:08.685426 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb978c-1a92-4f57-9e34-f39bb0c577ab None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:08.836705 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5924b7-3798-4914-a473-63b8b4c1a68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:08.837759 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5924b7-3798-4914-a473-63b8b4c1a68e admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:08.838171 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5924b7-3798-4914-a473-63b8b4c1a68e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:08.855030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5924b7-3798-4914-a473-63b8b4c1a68e admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:08.855030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5924b7-3798-4914-a473-63b8b4c1a68e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:08.855030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5924b7-3798-4914-a473-63b8b4c1a68e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:08.905642 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1109/2206] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:10:08 2022] POST /identity/v3/users => generated 293 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:10:08.912148 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19280640-696f-46c8-ad30-bd2e36048419 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:08.940296 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19280640-696f-46c8-ad30-bd2e36048419 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:08.959387 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514b61b-02b9-4fcb-a4b0-3d2da1e9651a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:08.960859 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514b61b-02b9-4fcb-a4b0-3d2da1e9651a admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:08.961436 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514b61b-02b9-4fcb-a4b0-3d2da1e9651a admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:08.962112 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514b61b-02b9-4fcb-a4b0-3d2da1e9651a admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:08.965171 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9514b61b-02b9-4fcb-a4b0-3d2da1e9651a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:08.982736 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9514b61b-02b9-4fcb-a4b0-3d2da1e9651a admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:09.031131 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1098/2207] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:10:09.038880 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5924b7-3798-4914-a473-63b8b4c1a68e admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:09.086176 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5924b7-3798-4914-a473-63b8b4c1a68e admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:09.107705 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe7a262-a433-4be5-919e-e63db676b601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:09.109376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe7a262-a433-4be5-919e-e63db676b601 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:09.109624 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe7a262-a433-4be5-919e-e63db676b601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:09.110249 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe7a262-a433-4be5-919e-e63db676b601 admin admin] PATH_INFO: `/v3/projects/2a959751ba6c420db35e31f466cfac66/users/e71aa70f82884bb0abbb66fcb3aac731/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:09.175820 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514b61b-02b9-4fcb-a4b0-3d2da1e9651a admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:09.178953 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-9514b61b-02b9-4fcb-a4b0-3d2da1e9651a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:09.199474 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe7a262-a433-4be5-919e-e63db676b601 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a959751ba6c420db35e31f466cfac66, user_id=e71aa70f82884bb0abbb66fcb3aac731, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:09.215306 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe7a262-a433-4be5-919e-e63db676b601 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:09.246484 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514b61b-02b9-4fcb-a4b0-3d2da1e9651a admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:09.289871 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1110/2208] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:10:09 2022] PUT /identity/v3/projects/2a959751ba6c420db35e31f466cfac66/users/e71aa70f82884bb0abbb66fcb3aac731/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:09.291689 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe7a262-a433-4be5-919e-e63db676b601 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:09.508930 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:09.513508 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ca21e-a5f9-4d82-8e79-f1a3a604b405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:09.514359 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ca21e-a5f9-4d82-8e79-f1a3a604b405 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:09.514558 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ca21e-a5f9-4d82-8e79-f1a3a604b405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:09.514779 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ca21e-a5f9-4d82-8e79-f1a3a604b405 admin admin] PATH_INFO: `/v3/projects/a687a9ada4494ab4b93aa989fe7b47a2` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:09.516677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ca21e-a5f9-4d82-8e79-f1a3a604b405 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a687a9ada4494ab4b93aa989fe7b47a2)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:09.518882 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ca21e-a5f9-4d82-8e79-f1a3a604b405 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:09.534618 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe7a262-a433-4be5-919e-e63db676b601 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:09.548601 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f4da1e-cfc7-444c-9fe2-a298e42fb3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:09.549278 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f4da1e-cfc7-444c-9fe2-a298e42fb3bf admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:09.549568 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f4da1e-cfc7-444c-9fe2-a298e42fb3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:09.549910 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f4da1e-cfc7-444c-9fe2-a298e42fb3bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:09.550682 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f4da1e-cfc7-444c-9fe2-a298e42fb3bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:09.552550 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f4da1e-cfc7-444c-9fe2-a298e42fb3bf admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:09.579603 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1111/2209] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:09 2022] GET /identity/v3/roles => generated 1420 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:10:09.585185 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f4da1e-cfc7-444c-9fe2-a298e42fb3bf admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:09.618589 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f4da1e-cfc7-444c-9fe2-a298e42fb3bf admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:09.660620 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4f7ca21e-a5f9-4d82-8e79-f1a3a604b405 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a687a9ada4494ab4b93aa989fe7b47a2', 'request_id': 'req-4f7ca21e-a5f9-4d82-8e79-f1a3a604b405'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:09.680997 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4f7ca21e-a5f9-4d82-8e79-f1a3a604b405 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a687a9ada4494ab4b93aa989fe7b47a2', 'request_id': 'req-4f7ca21e-a5f9-4d82-8e79-f1a3a604b405'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:09.700698 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4f7ca21e-a5f9-4d82-8e79-f1a3a604b405 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a687a9ada4494ab4b93aa989fe7b47a2', 'request_id': 'req-4f7ca21e-a5f9-4d82-8e79-f1a3a604b405'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:09.740297 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4f7ca21e-a5f9-4d82-8e79-f1a3a604b405 admin admin] The token cache is being invalidate because project a687a9ada4494ab4b93aa989fe7b47a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:10:09.740629 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4f7ca21e-a5f9-4d82-8e79-f1a3a604b405 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:09.742232 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4f7ca21e-a5f9-4d82-8e79-f1a3a604b405 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:09.746823 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1099/2210] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:10:09 2022] DELETE /identity/v3/projects/a687a9ada4494ab4b93aa989fe7b47a2 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:09.748019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ca21e-a5f9-4d82-8e79-f1a3a604b405 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:09.750165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-4f7ca21e-a5f9-4d82-8e79-f1a3a604b405 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:09.826336 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ece4a5-1046-40fb-9c4c-03a47ae2506d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:09.828104 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:09.829173 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ece4a5-1046-40fb-9c4c-03a47ae2506d admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:09.831342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ece4a5-1046-40fb-9c4c-03a47ae2506d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:09.832484 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ece4a5-1046-40fb-9c4c-03a47ae2506d admin admin] PATH_INFO: `/v3/projects/2a959751ba6c420db35e31f466cfac66/users/e71aa70f82884bb0abbb66fcb3aac731/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:09.848433 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ece4a5-1046-40fb-9c4c-03a47ae2506d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a959751ba6c420db35e31f466cfac66, user_id=e71aa70f82884bb0abbb66fcb3aac731, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:09.862812 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ece4a5-1046-40fb-9c4c-03a47ae2506d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:09.905930 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1112/2211] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:10:09 2022] PUT /identity/v3/projects/2a959751ba6c420db35e31f466cfac66/users/e71aa70f82884bb0abbb66fcb3aac731/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:09.926025 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09d410-40e9-4595-b9b3-8bd6de3772f2 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:09.930041 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09d410-40e9-4595-b9b3-8bd6de3772f2 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:09.930635 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09d410-40e9-4595-b9b3-8bd6de3772f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:10.027273 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-3e09d410-40e9-4595-b9b3-8bd6de3772f2 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:10.032928 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3e09d410-40e9-4595-b9b3-8bd6de3772f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:10.051420 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ca21e-a5f9-4d82-8e79-f1a3a604b405 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:10.285963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1113/2212] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:10.285963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f957464-3dd7-40d4-a5fd-5ffedcecdf2e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:10.285963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f957464-3dd7-40d4-a5fd-5ffedcecdf2e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:10.285963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f957464-3dd7-40d4-a5fd-5ffedcecdf2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:10.339498 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2e7fa-dbde-447e-8e34-33d1dd53045a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:10.340698 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2e7fa-dbde-447e-8e34-33d1dd53045a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:10.341627 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2e7fa-dbde-447e-8e34-33d1dd53045a admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:10.341627 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2e7fa-dbde-447e-8e34-33d1dd53045a admin admin] PATH_INFO: `/v3/users/aae0fa8c628e47a4b207ab4f281f271d` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:10.348162 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d2e7fa-dbde-447e-8e34-33d1dd53045a admin admin] RBAC: Authorizing `identity:delete_user(user_id=aae0fa8c628e47a4b207ab4f281f271d)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:10.363731 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d2e7fa-dbde-447e-8e34-33d1dd53045a admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:10.371891 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-3f957464-3dd7-40d4-a5fd-5ffedcecdf2e None None] MFA Rules not processed for user `e71aa70f82884bb0abbb66fcb3aac731`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:10.375654 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3f957464-3dd7-40d4-a5fd-5ffedcecdf2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:10.625787 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a6d2e7fa-dbde-447e-8e34-33d1dd53045a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aae0fa8c628e47a4b207ab4f281f271d'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:10.628317 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a6d2e7fa-dbde-447e-8e34-33d1dd53045a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aae0fa8c628e47a4b207ab4f281f271d'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:10.634341 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1114/2213] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:10 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:10.650295 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3e334-d72c-4015-8d16-a935ea77496c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:10.654226 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3e334-d72c-4015-8d16-a935ea77496c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:10.656571 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3e334-d72c-4015-8d16-a935ea77496c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:10.660699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a6d2e7fa-dbde-447e-8e34-33d1dd53045a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aae0fa8c628e47a4b207ab4f281f271d'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:10.708774 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a6d2e7fa-dbde-447e-8e34-33d1dd53045a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aae0fa8c628e47a4b207ab4f281f271d'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:10.725798 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:10.759660 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b6c3e334-d72c-4015-8d16-a935ea77496c None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:10.761272 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a6d2e7fa-dbde-447e-8e34-33d1dd53045a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aae0fa8c628e47a4b207ab4f281f271d'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:10.779487 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b6c3e334-d72c-4015-8d16-a935ea77496c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:10.798291 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1100/2214] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:10:09 2022] DELETE /identity/v3/users/aae0fa8c628e47a4b207ab4f281f271d => generated 0 bytes in 1050 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:10.799975 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2e7fa-dbde-447e-8e34-33d1dd53045a admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:10.802208 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a6d2e7fa-dbde-447e-8e34-33d1dd53045a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:10.818906 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1115/2215] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:10.821907 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96256d3f-15f4-4d4f-b8de-e9a6540963d7 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:10.823977 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96256d3f-15f4-4d4f-b8de-e9a6540963d7 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:10.824877 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96256d3f-15f4-4d4f-b8de-e9a6540963d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:10.872411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-96256d3f-15f4-4d4f-b8de-e9a6540963d7 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:10.874677 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-96256d3f-15f4-4d4f-b8de-e9a6540963d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:11.087228 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:11.102500 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1116/2216] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:11.109327 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96256d3f-15f4-4d4f-b8de-e9a6540963d7 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:11.110678 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2e7fa-dbde-447e-8e34-33d1dd53045a admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:11.128113 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e680f3d8-7566-427b-b489-4405c39bb575 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:11.129402 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680f3d8-7566-427b-b489-4405c39bb575 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:11.129968 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680f3d8-7566-427b-b489-4405c39bb575 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:11.130548 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680f3d8-7566-427b-b489-4405c39bb575 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:11.145090 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e680f3d8-7566-427b-b489-4405c39bb575 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:11.159552 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96256d3f-15f4-4d4f-b8de-e9a6540963d7 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:11.171053 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4a80e-4617-4088-afff-0161c3be84ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:11.172321 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4a80e-4617-4088-afff-0161c3be84ee admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:11.172905 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4a80e-4617-4088-afff-0161c3be84ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:11.173552 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4a80e-4617-4088-afff-0161c3be84ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:11.176366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad4a80e-4617-4088-afff-0161c3be84ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:10:11.177000 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad4a80e-4617-4088-afff-0161c3be84ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:11.180250 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad4a80e-4617-4088-afff-0161c3be84ee admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:11.230180 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1117/2217] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:10:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:10:11.248200 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4a80e-4617-4088-afff-0161c3be84ee admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:11.251481 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-aad4a80e-4617-4088-afff-0161c3be84ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:11.473134 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4a80e-4617-4088-afff-0161c3be84ee admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:11.490707 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181741bd-1a80-48b4-bff4-f8eac6820484 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:11.491489 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181741bd-1a80-48b4-bff4-f8eac6820484 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:11.491985 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181741bd-1a80-48b4-bff4-f8eac6820484 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:11.492368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181741bd-1a80-48b4-bff4-f8eac6820484 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:11.518870 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e680f3d8-7566-427b-b489-4405c39bb575 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:11.521785 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e680f3d8-7566-427b-b489-4405c39bb575 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:11.562643 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181741bd-1a80-48b4-bff4-f8eac6820484 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:11.565874 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181741bd-1a80-48b4-bff4-f8eac6820484 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:11.566711 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1101/2218] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 768 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:10:11.568524 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e680f3d8-7566-427b-b489-4405c39bb575 service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:11.608001 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1118/2219] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:10:11.621735 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e680f3d8-7566-427b-b489-4405c39bb575 service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:11.631202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71adff-f621-4f55-be11-b2d7d19a4efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:11.634519 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71adff-f621-4f55-be11-b2d7d19a4efd admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:11.635344 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71adff-f621-4f55-be11-b2d7d19a4efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:11.636147 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71adff-f621-4f55-be11-b2d7d19a4efd admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:11.638801 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71adff-f621-4f55-be11-b2d7d19a4efd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:11.642323 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71adff-f621-4f55-be11-b2d7d19a4efd admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:11.769234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1102/2220] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:10:11 2022] POST /identity/v3/projects => generated 373 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:10:11.776420 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181741bd-1a80-48b4-bff4-f8eac6820484 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:11.829369 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181741bd-1a80-48b4-bff4-f8eac6820484 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:12.061914 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b21d3-9d00-4eb4-834a-906d5ae72261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:12.063080 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b21d3-9d00-4eb4-834a-906d5ae72261 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:12.063589 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b21d3-9d00-4eb4-834a-906d5ae72261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:12.064055 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b21d3-9d00-4eb4-834a-906d5ae72261 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:12.067395 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b21d3-9d00-4eb4-834a-906d5ae72261 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:12.094106 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b21d3-9d00-4eb4-834a-906d5ae72261 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:12.196810 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1119/2221] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:10:11 2022] POST /identity/v3/users => generated 287 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:10:12.199697 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71adff-f621-4f55-be11-b2d7d19a4efd admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:12.231505 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71adff-f621-4f55-be11-b2d7d19a4efd admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:12.254959 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d5fbb-ed8d-47b3-9bcb-1c0f7ce8ca3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:12.260831 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d5fbb-ed8d-47b3-9bcb-1c0f7ce8ca3d admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:12.261511 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d5fbb-ed8d-47b3-9bcb-1c0f7ce8ca3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:12.264339 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d5fbb-ed8d-47b3-9bcb-1c0f7ce8ca3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:12.265513 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d5fbb-ed8d-47b3-9bcb-1c0f7ce8ca3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:12.267937 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d5fbb-ed8d-47b3-9bcb-1c0f7ce8ca3d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:12.313217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1103/2222] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:12 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:10:12.320547 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b21d3-9d00-4eb4-834a-906d5ae72261 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:12.377378 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b21d3-9d00-4eb4-834a-906d5ae72261 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:12.424664 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70eef55-7563-4f96-8d60-6b4d5f56a855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:12.425819 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70eef55-7563-4f96-8d60-6b4d5f56a855 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:12.426289 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70eef55-7563-4f96-8d60-6b4d5f56a855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:12.434594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70eef55-7563-4f96-8d60-6b4d5f56a855 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:12.436305 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70eef55-7563-4f96-8d60-6b4d5f56a855 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:12.439413 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70eef55-7563-4f96-8d60-6b4d5f56a855 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:12.489783 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1120/2223] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:12 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:10:12.490506 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d5fbb-ed8d-47b3-9bcb-1c0f7ce8ca3d admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:12.507779 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d5fbb-ed8d-47b3-9bcb-1c0f7ce8ca3d admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:12.539364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69befbcf-8a9e-49fb-a4e0-03c8cf038261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:12.539364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69befbcf-8a9e-49fb-a4e0-03c8cf038261 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:12.539364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69befbcf-8a9e-49fb-a4e0-03c8cf038261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:12.539364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69befbcf-8a9e-49fb-a4e0-03c8cf038261 admin admin] PATH_INFO: `/v3/projects/91dc237a97b94540a3653da8efccfa83/users/856cc3e170274ed8915e9621bbd7c105/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:12.691904 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c579e5-a022-4492-acdb-64fe17b6f274 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:12.692455 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c579e5-a022-4492-acdb-64fe17b6f274 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:12.692981 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c579e5-a022-4492-acdb-64fe17b6f274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:12.693297 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69befbcf-8a9e-49fb-a4e0-03c8cf038261 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91dc237a97b94540a3653da8efccfa83, user_id=856cc3e170274ed8915e9621bbd7c105, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:12.715448 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69befbcf-8a9e-49fb-a4e0-03c8cf038261 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:12.758386 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1104/2224] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:10:12 2022] PUT /identity/v3/projects/91dc237a97b94540a3653da8efccfa83/users/856cc3e170274ed8915e9621bbd7c105/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:10:12.782307 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc9cdd-0014-4ba6-aafe-dfd91cc79bb3 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:12.783345 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc9cdd-0014-4ba6-aafe-dfd91cc79bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:12.783972 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc9cdd-0014-4ba6-aafe-dfd91cc79bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:12.791577 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:12.851280 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d2c579e5-a022-4492-acdb-64fe17b6f274 None None] MFA Rules not processed for user `403896b926a1445b9895dc2d8d21854f`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:12.853460 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d2c579e5-a022-4492-acdb-64fe17b6f274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:12.954443 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-01cc9cdd-0014-4ba6-aafe-dfd91cc79bb3 None None] MFA Rules not processed for user `856cc3e170274ed8915e9621bbd7c105`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:12.968733 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-01cc9cdd-0014-4ba6-aafe-dfd91cc79bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:13.240502 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1121/2225] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 542 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:10:13.264976 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c579e5-a022-4492-acdb-64fe17b6f274 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:13.273003 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d2c579e5-a022-4492-acdb-64fe17b6f274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:13.306425 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1105/2226] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 527 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:13.342748 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc9cdd-0014-4ba6-aafe-dfd91cc79bb3 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:13.349452 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-01cc9cdd-0014-4ba6-aafe-dfd91cc79bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:13.546482 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c579e5-a022-4492-acdb-64fe17b6f274 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:13.557652 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09263273-d715-4a5e-a795-242d1873135e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:13.558659 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09263273-d715-4a5e-a795-242d1873135e service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:13.559392 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:13.577473 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09263273-d715-4a5e-a795-242d1873135e service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:13.579010 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09263273-d715-4a5e-a795-242d1873135e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:13.605609 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09263273-d715-4a5e-a795-242d1873135e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:13.611622 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09263273-d715-4a5e-a795-242d1873135e service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:13.629534 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1122/2227] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:13 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:10:13.630847 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc9cdd-0014-4ba6-aafe-dfd91cc79bb3 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:13.652920 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d43cd9-d4ed-44c3-95ba-1479a37ad9ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:13.654143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d43cd9-d4ed-44c3-95ba-1479a37ad9ea service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:13.654636 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d43cd9-d4ed-44c3-95ba-1479a37ad9ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:13.655474 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d43cd9-d4ed-44c3-95ba-1479a37ad9ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:13.706602 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d43cd9-d4ed-44c3-95ba-1479a37ad9ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:13.710718 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d43cd9-d4ed-44c3-95ba-1479a37ad9ea service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:13.745344 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1106/2228] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:10:13.813293 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09263273-d715-4a5e-a795-242d1873135e service nova] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:14.022672 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d43cd9-d4ed-44c3-95ba-1479a37ad9ea service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:14.025706 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-07d43cd9-d4ed-44c3-95ba-1479a37ad9ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:14.193285 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09263273-d715-4a5e-a795-242d1873135e service nova] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:14.215104 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbafd1cb-0372-4273-aa38-5a4ec94aa48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:14.215502 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafd1cb-0372-4273-aa38-5a4ec94aa48a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:14.215857 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafd1cb-0372-4273-aa38-5a4ec94aa48a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:14.216146 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafd1cb-0372-4273-aa38-5a4ec94aa48a admin admin] PATH_INFO: `/v3/users/856cc3e170274ed8915e9621bbd7c105` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:14.222687 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbafd1cb-0372-4273-aa38-5a4ec94aa48a admin admin] RBAC: Authorizing `identity:delete_user(user_id=856cc3e170274ed8915e9621bbd7c105)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:14.224840 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbafd1cb-0372-4273-aa38-5a4ec94aa48a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:14.302782 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:14.347937 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cbafd1cb-0372-4273-aa38-5a4ec94aa48a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '856cc3e170274ed8915e9621bbd7c105'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:14.348694 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cbafd1cb-0372-4273-aa38-5a4ec94aa48a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '856cc3e170274ed8915e9621bbd7c105'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:14.381133 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cbafd1cb-0372-4273-aa38-5a4ec94aa48a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '856cc3e170274ed8915e9621bbd7c105'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:14.424196 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cbafd1cb-0372-4273-aa38-5a4ec94aa48a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '856cc3e170274ed8915e9621bbd7c105'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:14.434035 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cbafd1cb-0372-4273-aa38-5a4ec94aa48a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '856cc3e170274ed8915e9621bbd7c105'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:14.497268 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1123/2229] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:10:13 2022] DELETE /identity/v3/users/856cc3e170274ed8915e9621bbd7c105 => generated 0 bytes in 685 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:14.526064 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c885edf-c6ad-4f78-b08e-ea789ac272c1 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:14.526520 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c885edf-c6ad-4f78-b08e-ea789ac272c1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:14.526711 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c885edf-c6ad-4f78-b08e-ea789ac272c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:14.574721 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d43cd9-d4ed-44c3-95ba-1479a37ad9ea service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:14.629014 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8c885edf-c6ad-4f78-b08e-ea789ac272c1 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:14.632848 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8c885edf-c6ad-4f78-b08e-ea789ac272c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:14.961227 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e73c6-0a08-4457-a762-b1c29483186f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:14.966360 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e73c6-0a08-4457-a762-b1c29483186f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:14.968202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e73c6-0a08-4457-a762-b1c29483186f admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:14.971652 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e73c6-0a08-4457-a762-b1c29483186f admin admin] PATH_INFO: `/v3/projects/fe63d5f4cc644128912d157da6ea8e41` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:14.978082 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e73c6-0a08-4457-a762-b1c29483186f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe63d5f4cc644128912d157da6ea8e41)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:14.988657 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e73c6-0a08-4457-a762-b1c29483186f admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:15.000220 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1124/2230] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 491 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:15.102044 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c885edf-c6ad-4f78-b08e-ea789ac272c1 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:15.104896 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8c885edf-c6ad-4f78-b08e-ea789ac272c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:15.287997 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-9e1e73c6-0a08-4457-a762-b1c29483186f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe63d5f4cc644128912d157da6ea8e41', 'request_id': 'req-9e1e73c6-0a08-4457-a762-b1c29483186f'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:15.340287 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-9e1e73c6-0a08-4457-a762-b1c29483186f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe63d5f4cc644128912d157da6ea8e41', 'request_id': 'req-9e1e73c6-0a08-4457-a762-b1c29483186f'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:15.379448 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-9e1e73c6-0a08-4457-a762-b1c29483186f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe63d5f4cc644128912d157da6ea8e41', 'request_id': 'req-9e1e73c6-0a08-4457-a762-b1c29483186f'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:15.511214 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-9e1e73c6-0a08-4457-a762-b1c29483186f admin admin] The token cache is being invalidate because project fe63d5f4cc644128912d157da6ea8e41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:10:15.511545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-9e1e73c6-0a08-4457-a762-b1c29483186f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:15.513175 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-9e1e73c6-0a08-4457-a762-b1c29483186f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:15.524170 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1107/2231] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:10:14 2022] DELETE /identity/v3/projects/fe63d5f4cc644128912d157da6ea8e41 => generated 0 bytes in 1495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:15.528570 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c885edf-c6ad-4f78-b08e-ea789ac272c1 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:15.533695 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b67426-16d5-4196-8f74-5a0b8f64788d None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:15.534377 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b67426-16d5-4196-8f74-5a0b8f64788d None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:15.537159 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b67426-16d5-4196-8f74-5a0b8f64788d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:15.656633 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-45b67426-16d5-4196-8f74-5a0b8f64788d None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:15.691105 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-45b67426-16d5-4196-8f74-5a0b8f64788d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:15.711599 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be911328-2773-4630-a4df-b5571b64cd30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:15.716773 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be911328-2773-4630-a4df-b5571b64cd30 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:15.733766 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be911328-2773-4630-a4df-b5571b64cd30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:15.735919 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be911328-2773-4630-a4df-b5571b64cd30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:15.742265 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-be911328-2773-4630-a4df-b5571b64cd30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:15.745824 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:15.789011 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:16.077591 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1108/2232] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 552 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:16.080296 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ef178-61fe-4c2d-81a4-d5a7c31a824d None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:16.080797 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ef178-61fe-4c2d-81a4-d5a7c31a824d None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:16.081317 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ef178-61fe-4c2d-81a4-d5a7c31a824d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:16.089095 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be911328-2773-4630-a4df-b5571b64cd30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:16.091690 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be911328-2773-4630-a4df-b5571b64cd30 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:16.149731 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1125/2233] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 1049 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:10:16.151223 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69caab8-2459-46f5-a616-62cb0164485a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:16.151529 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69caab8-2459-46f5-a616-62cb0164485a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:16.151795 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69caab8-2459-46f5-a616-62cb0164485a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:16.241798 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-f69caab8-2459-46f5-a616-62cb0164485a None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:16.248925 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f69caab8-2459-46f5-a616-62cb0164485a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:16.286074 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-869ef178-61fe-4c2d-81a4-d5a7c31a824d None None] MFA Rules not processed for user `ba2b258c27f44ba39e76eff0a8602a95`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:16.288149 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-869ef178-61fe-4c2d-81a4-d5a7c31a824d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:16.295313 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1126/2234] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:16.304915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69caab8-2459-46f5-a616-62cb0164485a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:16.337087 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69caab8-2459-46f5-a616-62cb0164485a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:16.360068 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92403d08-b926-4e48-905a-ce9dbee46138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:16.364045 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92403d08-b926-4e48-905a-ce9dbee46138 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:16.364660 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92403d08-b926-4e48-905a-ce9dbee46138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:16.365105 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92403d08-b926-4e48-905a-ce9dbee46138 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:16.372553 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92403d08-b926-4e48-905a-ce9dbee46138 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:10:16.373134 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92403d08-b926-4e48-905a-ce9dbee46138 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:16.382373 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92403d08-b926-4e48-905a-ce9dbee46138 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:16.451928 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1127/2235] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:10:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:10:16.468790 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92403d08-b926-4e48-905a-ce9dbee46138 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:16.476851 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-92403d08-b926-4e48-905a-ce9dbee46138 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:16.561028 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92403d08-b926-4e48-905a-ce9dbee46138 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:16.588436 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0db21-fa33-4d1b-bd4f-3eca359756b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:16.592384 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0db21-fa33-4d1b-bd4f-3eca359756b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:16.592753 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0db21-fa33-4d1b-bd4f-3eca359756b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:16.593203 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0db21-fa33-4d1b-bd4f-3eca359756b0 admin admin] PATH_INFO: `/v3/users/e0ce1e490e5741bdbe1cee1c0426ae1e` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:16.605394 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0db21-fa33-4d1b-bd4f-3eca359756b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0ce1e490e5741bdbe1cee1c0426ae1e)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:16.607483 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0db21-fa33-4d1b-bd4f-3eca359756b0 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:16.733755 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1109/2236] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:16 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 656 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:16.746360 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ef178-61fe-4c2d-81a4-d5a7c31a824d None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:16.767418 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-aba0db21-fa33-4d1b-bd4f-3eca359756b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0ce1e490e5741bdbe1cee1c0426ae1e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:16.769168 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-aba0db21-fa33-4d1b-bd4f-3eca359756b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0ce1e490e5741bdbe1cee1c0426ae1e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:16.794032 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ef178-61fe-4c2d-81a4-d5a7c31a824d None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:16.812561 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d34f62a-a4c3-4d22-b037-5041967af882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:16.820043 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34f62a-a4c3-4d22-b037-5041967af882 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:16.820789 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34f62a-a4c3-4d22-b037-5041967af882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:16.821398 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34f62a-a4c3-4d22-b037-5041967af882 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:16.823065 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d34f62a-a4c3-4d22-b037-5041967af882 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:16.828903 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d34f62a-a4c3-4d22-b037-5041967af882 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:16.830080 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-aba0db21-fa33-4d1b-bd4f-3eca359756b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0ce1e490e5741bdbe1cee1c0426ae1e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:16.847417 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-aba0db21-fa33-4d1b-bd4f-3eca359756b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0ce1e490e5741bdbe1cee1c0426ae1e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:16.852260 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-aba0db21-fa33-4d1b-bd4f-3eca359756b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0ce1e490e5741bdbe1cee1c0426ae1e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:16.923450 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1128/2237] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:10:16 2022] DELETE /identity/v3/users/e0ce1e490e5741bdbe1cee1c0426ae1e => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:16.929003 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0db21-fa33-4d1b-bd4f-3eca359756b0 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:16.932095 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-aba0db21-fa33-4d1b-bd4f-3eca359756b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:16.995530 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1110/2238] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:10:16 2022] POST /identity/v3/projects => generated 377 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:10:17.008136 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bdffe-568f-486a-8af2-ca08332def96 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:17.008717 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bdffe-568f-486a-8af2-ca08332def96 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:17.011013 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bdffe-568f-486a-8af2-ca08332def96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:17.073296 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-693bdffe-568f-486a-8af2-ca08332def96 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:17.075034 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-693bdffe-568f-486a-8af2-ca08332def96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:17.228129 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0db21-fa33-4d1b-bd4f-3eca359756b0 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:17.259034 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1111/2239] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:17.261327 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693bdffe-568f-486a-8af2-ca08332def96 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:17.275510 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-693bdffe-568f-486a-8af2-ca08332def96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:17.278628 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:17.332424 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693bdffe-568f-486a-8af2-ca08332def96 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:17.351328 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e3d7bc-2dfe-40c7-9ce2-de16474a841d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:17.352429 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3d7bc-2dfe-40c7-9ce2-de16474a841d admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:17.359146 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3d7bc-2dfe-40c7-9ce2-de16474a841d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:17.360722 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3d7bc-2dfe-40c7-9ce2-de16474a841d admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:17.362258 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e3d7bc-2dfe-40c7-9ce2-de16474a841d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:17.365408 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e3d7bc-2dfe-40c7-9ce2-de16474a841d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:17.383347 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856472f-ca8c-40da-aba6-5b5ab7afe37c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7317c8829cb4fa1b30711f1db575cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:17.384619 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856472f-ca8c-40da-aba6-5b5ab7afe37c service glance] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:17.385163 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856472f-ca8c-40da-aba6-5b5ab7afe37c service glance] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:17.385671 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856472f-ca8c-40da-aba6-5b5ab7afe37c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:17.389896 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9856472f-ca8c-40da-aba6-5b5ab7afe37c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:17.432032 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1112/2240] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:10:17 2022] POST /identity/v3/users => generated 289 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:10:17.432032 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e3d7bc-2dfe-40c7-9ce2-de16474a841d admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:17.437634 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-49e3d7bc-2dfe-40c7-9ce2-de16474a841d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:17.452722 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:17.594800 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9856472f-ca8c-40da-aba6-5b5ab7afe37c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:17.597676 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9856472f-ca8c-40da-aba6-5b5ab7afe37c service glance] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:17.622417 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1129/2241] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 698 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:10:17.624862 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856472f-ca8c-40da-aba6-5b5ab7afe37c service glance] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:17.656326 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e3d7bc-2dfe-40c7-9ce2-de16474a841d admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:17.674194 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e2ba7-a69c-4a80-94a4-14e2819d6579 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:17.676011 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e2ba7-a69c-4a80-94a4-14e2819d6579 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:17.676564 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e2ba7-a69c-4a80-94a4-14e2819d6579 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:17.676997 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e2ba7-a69c-4a80-94a4-14e2819d6579 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:17.694658 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856472f-ca8c-40da-aba6-5b5ab7afe37c service glance] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:17.700735 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e2ba7-a69c-4a80-94a4-14e2819d6579 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:17.708933 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e2ba7-a69c-4a80-94a4-14e2819d6579 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:17.710149 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2008d-fdfe-4a76-9671-28cfcc85220c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:17.716170 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2008d-fdfe-4a76-9671-28cfcc85220c admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:17.716728 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2008d-fdfe-4a76-9671-28cfcc85220c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:17.717180 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2008d-fdfe-4a76-9671-28cfcc85220c admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:17.718397 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c2008d-fdfe-4a76-9671-28cfcc85220c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:17.731507 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c2008d-fdfe-4a76-9671-28cfcc85220c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:17.767065 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1113/2242] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:10:17.778081 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e2ba7-a69c-4a80-94a4-14e2819d6579 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:17.779118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1130/2243] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:10:17.818120 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e2ba7-a69c-4a80-94a4-14e2819d6579 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:17.841360 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aca77da-3ab0-49d8-9d4b-4b824d4a478c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:17.842097 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca77da-3ab0-49d8-9d4b-4b824d4a478c admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:17.842547 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca77da-3ab0-49d8-9d4b-4b824d4a478c admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:17.843004 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca77da-3ab0-49d8-9d4b-4b824d4a478c admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:17.843980 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aca77da-3ab0-49d8-9d4b-4b824d4a478c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:17.889961 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aca77da-3ab0-49d8-9d4b-4b824d4a478c admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:17.990385 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1114/2244] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:10:17.996057 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2008d-fdfe-4a76-9671-28cfcc85220c admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:18.022083 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2008d-fdfe-4a76-9671-28cfcc85220c admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:18.033376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad0186-d00e-4ac6-816d-f88b49d58d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:18.042116 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad0186-d00e-4ac6-816d-f88b49d58d6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:18.043185 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad0186-d00e-4ac6-816d-f88b49d58d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:18.043715 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad0186-d00e-4ac6-816d-f88b49d58d6f admin admin] PATH_INFO: `/v3/projects/f0a6c07d69ef4cf4a545f48e7079044d/users/8e30910692b4474092f1585a671e2f41/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:18.115247 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad0186-d00e-4ac6-816d-f88b49d58d6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0a6c07d69ef4cf4a545f48e7079044d, user_id=8e30910692b4474092f1585a671e2f41, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:18.125737 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad0186-d00e-4ac6-816d-f88b49d58d6f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:18.172531 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1131/2245] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:10:17 2022] PUT /identity/v3/projects/f0a6c07d69ef4cf4a545f48e7079044d/users/8e30910692b4474092f1585a671e2f41/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:10:18.186222 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f43be9-9a84-4084-af7c-c08c84b7c44d None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:18.187944 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f43be9-9a84-4084-af7c-c08c84b7c44d None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:18.189545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f43be9-9a84-4084-af7c-c08c84b7c44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:18.260879 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97895c-9193-4fd0-83de-6eefa09c4910 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:18.262021 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-40f43be9-9a84-4084-af7c-c08c84b7c44d None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:18.263212 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97895c-9193-4fd0-83de-6eefa09c4910 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:18.264917 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97895c-9193-4fd0-83de-6eefa09c4910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:18.266173 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-40f43be9-9a84-4084-af7c-c08c84b7c44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:18.318931 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8d97895c-9193-4fd0-83de-6eefa09c4910 None None] MFA Rules not processed for user `e07e83c0bdba47d39c09308eba1bdaae`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:18.320717 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8d97895c-9193-4fd0-83de-6eefa09c4910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:18.495857 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1115/2246] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:18.497673 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f43be9-9a84-4084-af7c-c08c84b7c44d None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:18.502258 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-40f43be9-9a84-4084-af7c-c08c84b7c44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:18.546368 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f43be9-9a84-4084-af7c-c08c84b7c44d None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:18.547028 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1132/2247] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Aug 1 14:10:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:18.558778 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97895c-9193-4fd0-83de-6eefa09c4910 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:18.563702 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74b577-1acf-463d-87b9-63ee638255f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:18.567385 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74b577-1acf-463d-87b9-63ee638255f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:18.570869 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74b577-1acf-463d-87b9-63ee638255f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:18.571636 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74b577-1acf-463d-87b9-63ee638255f9 admin admin] PATH_INFO: `/v3/projects/91dc237a97b94540a3653da8efccfa83` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:18.575602 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa74b577-1acf-463d-87b9-63ee638255f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91dc237a97b94540a3653da8efccfa83)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:18.582973 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa74b577-1acf-463d-87b9-63ee638255f9 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:18.684740 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:18.716740 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97895c-9193-4fd0-83de-6eefa09c4910 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:18.769876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-fa74b577-1acf-463d-87b9-63ee638255f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91dc237a97b94540a3653da8efccfa83', 'request_id': 'req-fa74b577-1acf-463d-87b9-63ee638255f9'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:18.803725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-fa74b577-1acf-463d-87b9-63ee638255f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91dc237a97b94540a3653da8efccfa83', 'request_id': 'req-fa74b577-1acf-463d-87b9-63ee638255f9'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:18.832794 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-fa74b577-1acf-463d-87b9-63ee638255f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91dc237a97b94540a3653da8efccfa83', 'request_id': 'req-fa74b577-1acf-463d-87b9-63ee638255f9'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:18.844581 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7771f-fc04-4a91-b223-3c90c8905513 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:18.845281 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7771f-fc04-4a91-b223-3c90c8905513 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:18.845878 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7771f-fc04-4a91-b223-3c90c8905513 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:18.846258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7771f-fc04-4a91-b223-3c90c8905513 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:18.869836 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-fa74b577-1acf-463d-87b9-63ee638255f9 admin admin] The token cache is being invalidate because project 91dc237a97b94540a3653da8efccfa83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:10:18.870443 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-fa74b577-1acf-463d-87b9-63ee638255f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:18.875114 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-fa74b577-1acf-463d-87b9-63ee638255f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:18.878540 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1116/2248] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:10:18 2022] DELETE /identity/v3/projects/91dc237a97b94540a3653da8efccfa83 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:18.881036 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74b577-1acf-463d-87b9-63ee638255f9 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:18.890666 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-fa74b577-1acf-463d-87b9-63ee638255f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:18.975063 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:19.080742 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c7771f-fc04-4a91-b223-3c90c8905513 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:19.086995 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c7771f-fc04-4a91-b223-3c90c8905513 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:19.145712 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1133/2249] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:10:19.148427 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eafb6c0-c5e3-44fd-8e97-c468327dab0b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:19.162385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eafb6c0-c5e3-44fd-8e97-c468327dab0b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:19.164586 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eafb6c0-c5e3-44fd-8e97-c468327dab0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:19.225231 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74b577-1acf-463d-87b9-63ee638255f9 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:19.232604 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7427f-b99e-466f-b93d-79c8e39e72f1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cda7f365ba0405daaf86870960f12d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:19.234016 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7427f-b99e-466f-b93d-79c8e39e72f1 service swift] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:19.236923 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7427f-b99e-466f-b93d-79c8e39e72f1 service swift] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:19.238549 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7427f-b99e-466f-b93d-79c8e39e72f1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:19.241202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-60a7427f-b99e-466f-b93d-79c8e39e72f1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:19.249482 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-2eafb6c0-c5e3-44fd-8e97-c468327dab0b None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:19.251718 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2eafb6c0-c5e3-44fd-8e97-c468327dab0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:19.303566 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1134/2250] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:19.306412 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722c627-dd01-4cbc-94ac-304a0b8cd3ce None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:19.306876 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722c627-dd01-4cbc-94ac-304a0b8cd3ce None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:19.307341 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722c627-dd01-4cbc-94ac-304a0b8cd3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:19.388743 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b722c627-dd01-4cbc-94ac-304a0b8cd3ce None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:19.390622 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b722c627-dd01-4cbc-94ac-304a0b8cd3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:19.444335 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1135/2251] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:19.451198 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722c627-dd01-4cbc-94ac-304a0b8cd3ce None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:19.479495 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722c627-dd01-4cbc-94ac-304a0b8cd3ce None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:19.494375 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc53df0-e7b2-4fca-8d2e-4164a9c75d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:19.495509 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc53df0-e7b2-4fca-8d2e-4164a9c75d82 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:19.495939 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc53df0-e7b2-4fca-8d2e-4164a9c75d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:19.496386 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc53df0-e7b2-4fca-8d2e-4164a9c75d82 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:19.497525 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc53df0-e7b2-4fca-8d2e-4164a9c75d82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:10:19.497988 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc53df0-e7b2-4fca-8d2e-4164a9c75d82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:19.499091 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7427f-b99e-466f-b93d-79c8e39e72f1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:19.500676 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc53df0-e7b2-4fca-8d2e-4164a9c75d82 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:19.500892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7427f-b99e-466f-b93d-79c8e39e72f1 service swift] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:19.512805 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1117/2252] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 14:10:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 633 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:10:19.558466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1136/2253] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:10:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:10:19.560210 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7427f-b99e-466f-b93d-79c8e39e72f1 service swift] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:19.579179 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7427f-b99e-466f-b93d-79c8e39e72f1 service swift] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:19.588847 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdbb997-3106-418f-864a-ef7105d3ff8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:19.590527 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdbb997-3106-418f-864a-ef7105d3ff8f admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:19.591354 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdbb997-3106-418f-864a-ef7105d3ff8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:19.592143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdbb997-3106-418f-864a-ef7105d3ff8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:19.593500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdbb997-3106-418f-864a-ef7105d3ff8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:19.598343 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdbb997-3106-418f-864a-ef7105d3ff8f admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:19.661924 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1118/2254] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:10:19 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:10:19.668072 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc53df0-e7b2-4fca-8d2e-4164a9c75d82 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:19.712839 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc53df0-e7b2-4fca-8d2e-4164a9c75d82 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:19.957655 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe397b-68af-4023-9119-e4984fac8bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:19.959334 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe397b-68af-4023-9119-e4984fac8bb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:19.959838 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe397b-68af-4023-9119-e4984fac8bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:19.960544 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe397b-68af-4023-9119-e4984fac8bb9 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:19.961659 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe397b-68af-4023-9119-e4984fac8bb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:19.965557 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe397b-68af-4023-9119-e4984fac8bb9 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:20.057157 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1137/2255] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:10:19 2022] POST /identity/v3/users => generated 287 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:10:20.075667 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdbb997-3106-418f-864a-ef7105d3ff8f admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:20.140166 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdbb997-3106-418f-864a-ef7105d3ff8f admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:20.183399 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eca13a-6a41-41e8-9a9f-96b5c2d6ed99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:20.187707 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eca13a-6a41-41e8-9a9f-96b5c2d6ed99 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:20.193969 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eca13a-6a41-41e8-9a9f-96b5c2d6ed99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:20.194624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eca13a-6a41-41e8-9a9f-96b5c2d6ed99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:20.203770 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eca13a-6a41-41e8-9a9f-96b5c2d6ed99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:20.217015 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eca13a-6a41-41e8-9a9f-96b5c2d6ed99 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:20.302703 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1119/2256] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:10:20.304004 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe397b-68af-4023-9119-e4984fac8bb9 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:20.372169 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe397b-68af-4023-9119-e4984fac8bb9 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:20.397629 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb006f-ca2e-4fc0-82d6-30f790e56761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:20.401218 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb006f-ca2e-4fc0-82d6-30f790e56761 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:20.401652 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb006f-ca2e-4fc0-82d6-30f790e56761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:20.408872 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb006f-ca2e-4fc0-82d6-30f790e56761 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:20.410115 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb006f-ca2e-4fc0-82d6-30f790e56761 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:20.417227 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb006f-ca2e-4fc0-82d6-30f790e56761 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:20.434367 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:20.454544 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1138/2257] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:10:20.462251 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eca13a-6a41-41e8-9a9f-96b5c2d6ed99 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:20.480533 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb006f-ca2e-4fc0-82d6-30f790e56761 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:20.486708 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e5eb006f-ca2e-4fc0-82d6-30f790e56761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:20.510847 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eca13a-6a41-41e8-9a9f-96b5c2d6ed99 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:20.532257 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7465ce5-a0cb-4bbf-9f4a-f9f02b3d0968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:20.541315 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb006f-ca2e-4fc0-82d6-30f790e56761 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:20.545777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a82f38-7702-4d65-8c1f-d9b3948caf8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:20.547376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a82f38-7702-4d65-8c1f-d9b3948caf8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:20.547664 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a82f38-7702-4d65-8c1f-d9b3948caf8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:20.548366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a82f38-7702-4d65-8c1f-d9b3948caf8e admin admin] PATH_INFO: `/v3/projects/3eba5c871a224532bb7a1f48891d3f50` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:20.551119 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a82f38-7702-4d65-8c1f-d9b3948caf8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3eba5c871a224532bb7a1f48891d3f50)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:20.557760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a82f38-7702-4d65-8c1f-d9b3948caf8e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:20.561537 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7465ce5-a0cb-4bbf-9f4a-f9f02b3d0968 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:20.561809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7465ce5-a0cb-4bbf-9f4a-f9f02b3d0968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:20.565389 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7465ce5-a0cb-4bbf-9f4a-f9f02b3d0968 admin admin] PATH_INFO: `/v3/projects/6bfbca571c4b4199a577322e866dfae9/users/b4ae8decd5354231bebc7bfda44d3285/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:20.620887 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7465ce5-a0cb-4bbf-9f4a-f9f02b3d0968 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bfbca571c4b4199a577322e866dfae9, user_id=b4ae8decd5354231bebc7bfda44d3285, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:20.627486 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7465ce5-a0cb-4bbf-9f4a-f9f02b3d0968 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:20.682205 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1120/2258] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:10:20 2022] PUT /identity/v3/projects/6bfbca571c4b4199a577322e866dfae9/users/b4ae8decd5354231bebc7bfda44d3285/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:20.691894 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6642bfd-c5f1-4e20-ba18-37cfdbbe5e09 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:20.692471 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6642bfd-c5f1-4e20-ba18-37cfdbbe5e09 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:20.693017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6642bfd-c5f1-4e20-ba18-37cfdbbe5e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:20.806064 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-a6642bfd-c5f1-4e20-ba18-37cfdbbe5e09 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:20.808733 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a6642bfd-c5f1-4e20-ba18-37cfdbbe5e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:20.811969 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-00a82f38-7702-4d65-8c1f-d9b3948caf8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3eba5c871a224532bb7a1f48891d3f50', 'request_id': 'req-00a82f38-7702-4d65-8c1f-d9b3948caf8e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:20.825388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-00a82f38-7702-4d65-8c1f-d9b3948caf8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3eba5c871a224532bb7a1f48891d3f50', 'request_id': 'req-00a82f38-7702-4d65-8c1f-d9b3948caf8e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:20.844610 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-00a82f38-7702-4d65-8c1f-d9b3948caf8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3eba5c871a224532bb7a1f48891d3f50', 'request_id': 'req-00a82f38-7702-4d65-8c1f-d9b3948caf8e'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:20.881242 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-00a82f38-7702-4d65-8c1f-d9b3948caf8e admin admin] The token cache is being invalidate because project 3eba5c871a224532bb7a1f48891d3f50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:10:20.882391 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-00a82f38-7702-4d65-8c1f-d9b3948caf8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:20.884652 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-00a82f38-7702-4d65-8c1f-d9b3948caf8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:20.896832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1139/2259] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:10:20 2022] DELETE /identity/v3/projects/3eba5c871a224532bb7a1f48891d3f50 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:20.900649 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a82f38-7702-4d65-8c1f-d9b3948caf8e admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:20.904164 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-00a82f38-7702-4d65-8c1f-d9b3948caf8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:21.009181 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1121/2260] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:10:21.022718 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6642bfd-c5f1-4e20-ba18-37cfdbbe5e09 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:21.024494 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a6642bfd-c5f1-4e20-ba18-37cfdbbe5e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:21.031723 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:21.200845 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a82f38-7702-4d65-8c1f-d9b3948caf8e admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:21.217422 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5942c-a7a8-43dc-9ff4-7b2db1a6fef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:21.222043 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5942c-a7a8-43dc-9ff4-7b2db1a6fef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:21.224192 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5942c-a7a8-43dc-9ff4-7b2db1a6fef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:21.227042 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5942c-a7a8-43dc-9ff4-7b2db1a6fef1 admin admin] PATH_INFO: `/v3/users/e71aa70f82884bb0abbb66fcb3aac731` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:21.235654 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5942c-a7a8-43dc-9ff4-7b2db1a6fef1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e71aa70f82884bb0abbb66fcb3aac731)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:21.241990 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5942c-a7a8-43dc-9ff4-7b2db1a6fef1 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:21.334083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6642bfd-c5f1-4e20-ba18-37cfdbbe5e09 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:21.357020 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0cfe3a-c7dd-4efc-b6f3-ef9567ab2c1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:21.362144 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cfe3a-c7dd-4efc-b6f3-ef9567ab2c1b service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:21.363568 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cfe3a-c7dd-4efc-b6f3-ef9567ab2c1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:21.366169 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cfe3a-c7dd-4efc-b6f3-ef9567ab2c1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:21.409127 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0cfe3a-c7dd-4efc-b6f3-ef9567ab2c1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:21.420939 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0cfe3a-c7dd-4efc-b6f3-ef9567ab2c1b service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:21.456817 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ddd5942c-a7a8-43dc-9ff4-7b2db1a6fef1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e71aa70f82884bb0abbb66fcb3aac731'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:21.458925 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ddd5942c-a7a8-43dc-9ff4-7b2db1a6fef1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e71aa70f82884bb0abbb66fcb3aac731'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:21.462540 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1122/2261] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:10:21.464472 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f38c47-9eaa-4d78-8f95-e27f8fc1eeaa None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:21.466421 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f38c47-9eaa-4d78-8f95-e27f8fc1eeaa None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:21.468848 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f38c47-9eaa-4d78-8f95-e27f8fc1eeaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:21.501152 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ddd5942c-a7a8-43dc-9ff4-7b2db1a6fef1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e71aa70f82884bb0abbb66fcb3aac731'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:21.525411 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-b0f38c47-9eaa-4d78-8f95-e27f8fc1eeaa None None] MFA Rules not processed for user `403896b926a1445b9895dc2d8d21854f`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:21.527307 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ddd5942c-a7a8-43dc-9ff4-7b2db1a6fef1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e71aa70f82884bb0abbb66fcb3aac731'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:21.531634 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b0f38c47-9eaa-4d78-8f95-e27f8fc1eeaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:21.532182 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ddd5942c-a7a8-43dc-9ff4-7b2db1a6fef1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e71aa70f82884bb0abbb66fcb3aac731'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:21.619125 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1140/2262] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:10:20 2022] DELETE /identity/v3/users/e71aa70f82884bb0abbb66fcb3aac731 => generated 0 bytes in 719 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:21.788860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1123/2263] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:23.515123 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5942c-a7a8-43dc-9ff4-7b2db1a6fef1 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:23.520511 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ddd5942c-a7a8-43dc-9ff4-7b2db1a6fef1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:23.579653 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:23.950744 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5942c-a7a8-43dc-9ff4-7b2db1a6fef1 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:23.973205 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838b8b0-557f-425f-97bc-fb315115777f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:23.974365 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838b8b0-557f-425f-97bc-fb315115777f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:23.974855 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838b8b0-557f-425f-97bc-fb315115777f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:23.975367 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838b8b0-557f-425f-97bc-fb315115777f admin admin] PATH_INFO: `/v3/projects/2a959751ba6c420db35e31f466cfac66` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:23.984331 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838b8b0-557f-425f-97bc-fb315115777f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a959751ba6c420db35e31f466cfac66)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:23.998003 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838b8b0-557f-425f-97bc-fb315115777f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:24.271654 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6838b8b0-557f-425f-97bc-fb315115777f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a959751ba6c420db35e31f466cfac66', 'request_id': 'req-6838b8b0-557f-425f-97bc-fb315115777f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:24.298577 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6838b8b0-557f-425f-97bc-fb315115777f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a959751ba6c420db35e31f466cfac66', 'request_id': 'req-6838b8b0-557f-425f-97bc-fb315115777f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:24.351103 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6838b8b0-557f-425f-97bc-fb315115777f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a959751ba6c420db35e31f466cfac66', 'request_id': 'req-6838b8b0-557f-425f-97bc-fb315115777f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:24.433848 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6838b8b0-557f-425f-97bc-fb315115777f admin admin] The token cache is being invalidate because project 2a959751ba6c420db35e31f466cfac66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:10:24.445018 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6838b8b0-557f-425f-97bc-fb315115777f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:24.447880 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6838b8b0-557f-425f-97bc-fb315115777f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:24.459757 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1141/2264] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:10:23 2022] DELETE /identity/v3/projects/2a959751ba6c420db35e31f466cfac66 => generated 0 bytes in 946 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:24.464944 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6a90da-9ada-4d33-b54b-a87f3ec17b39 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:24.466038 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6a90da-9ada-4d33-b54b-a87f3ec17b39 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:24.466543 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6a90da-9ada-4d33-b54b-a87f3ec17b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:24.604863 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-ce6a90da-9ada-4d33-b54b-a87f3ec17b39 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:24.606959 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ce6a90da-9ada-4d33-b54b-a87f3ec17b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:25.265237 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89738d1-aa5a-4364-a6f6-fdb373e72dcc None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:25.266943 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1124/2265] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 801 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:25.267970 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89738d1-aa5a-4364-a6f6-fdb373e72dcc None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:25.268536 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89738d1-aa5a-4364-a6f6-fdb373e72dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:25.367257 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d89738d1-aa5a-4364-a6f6-fdb373e72dcc None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:25.370298 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d89738d1-aa5a-4364-a6f6-fdb373e72dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:25.450497 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6a90da-9ada-4d33-b54b-a87f3ec17b39 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:25.451517 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1142/2266] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:25.510672 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6a90da-9ada-4d33-b54b-a87f3ec17b39 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:25.573357 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f082ee-4b72-4be5-a565-284e8e03e20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:25.574369 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f082ee-4b72-4be5-a565-284e8e03e20f admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:25.578482 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f082ee-4b72-4be5-a565-284e8e03e20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:25.579169 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f082ee-4b72-4be5-a565-284e8e03e20f admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:25.582223 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f082ee-4b72-4be5-a565-284e8e03e20f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:10:25.584664 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f082ee-4b72-4be5-a565-284e8e03e20f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:25.594267 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f082ee-4b72-4be5-a565-284e8e03e20f admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:25.678814 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89738d1-aa5a-4364-a6f6-fdb373e72dcc None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:25.697614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1125/2267] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:10:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:10:25.766782 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89738d1-aa5a-4364-a6f6-fdb373e72dcc None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:25.801588 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca7f31-cf9c-45ca-987d-5680869873da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:25.804581 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca7f31-cf9c-45ca-987d-5680869873da admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:25.808225 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca7f31-cf9c-45ca-987d-5680869873da admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:25.812051 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca7f31-cf9c-45ca-987d-5680869873da admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:25.819191 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca7f31-cf9c-45ca-987d-5680869873da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:25.831983 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca7f31-cf9c-45ca-987d-5680869873da admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:25.967292 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1143/2268] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:10:25 2022] POST /identity/v3/projects => generated 371 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:10:25.975945 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f082ee-4b72-4be5-a565-284e8e03e20f admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:26.034096 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f082ee-4b72-4be5-a565-284e8e03e20f admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:26.331925 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:26.493548 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1374899e-3540-43ff-a413-5e4b7b45a12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:26.496627 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374899e-3540-43ff-a413-5e4b7b45a12f admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:26.501084 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374899e-3540-43ff-a413-5e4b7b45a12f admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:26.501533 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374899e-3540-43ff-a413-5e4b7b45a12f admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:26.507049 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1374899e-3540-43ff-a413-5e4b7b45a12f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:26.512070 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1374899e-3540-43ff-a413-5e4b7b45a12f admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:26.661701 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1126/2269] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:10:25 2022] POST /identity/v3/users => generated 286 bytes in 687 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:10:26.667664 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca7f31-cf9c-45ca-987d-5680869873da admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:26.705256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca7f31-cf9c-45ca-987d-5680869873da admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:26.719840 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87380548-34ad-48c0-b77b-bf3fc7b9989f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:26.721388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87380548-34ad-48c0-b77b-bf3fc7b9989f admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:26.726956 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87380548-34ad-48c0-b77b-bf3fc7b9989f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:26.727817 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87380548-34ad-48c0-b77b-bf3fc7b9989f admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:26.729888 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87380548-34ad-48c0-b77b-bf3fc7b9989f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:26.736797 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87380548-34ad-48c0-b77b-bf3fc7b9989f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:26.788664 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1144/2270] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:10:26.795093 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1374899e-3540-43ff-a413-5e4b7b45a12f admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:26.853404 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1374899e-3540-43ff-a413-5e4b7b45a12f admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:26.874924 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02810d78-a69f-4f62-a9fd-9290f36521f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:26.879476 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02810d78-a69f-4f62-a9fd-9290f36521f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:26.879966 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02810d78-a69f-4f62-a9fd-9290f36521f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:26.884437 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02810d78-a69f-4f62-a9fd-9290f36521f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:26.886375 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02810d78-a69f-4f62-a9fd-9290f36521f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:26.900709 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02810d78-a69f-4f62-a9fd-9290f36521f5 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:26.984912 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1127/2271] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:10:26.995360 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87380548-34ad-48c0-b77b-bf3fc7b9989f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:27.035217 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87380548-34ad-48c0-b77b-bf3fc7b9989f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:27.058857 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b9244-2b17-4d97-af61-29a3389f480a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:27.059990 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b9244-2b17-4d97-af61-29a3389f480a admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:27.060488 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b9244-2b17-4d97-af61-29a3389f480a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:27.062085 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b9244-2b17-4d97-af61-29a3389f480a admin admin] PATH_INFO: `/v3/projects/803bbc59642547fb932be578cb70ac3f/users/824b6455aab74659a9fb1e5e8f7c2965/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:27.125932 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0b9244-2b17-4d97-af61-29a3389f480a admin admin] RBAC: Authorizing `identity:create_grant(project_id=803bbc59642547fb932be578cb70ac3f, user_id=824b6455aab74659a9fb1e5e8f7c2965, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:27.130137 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0b9244-2b17-4d97-af61-29a3389f480a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:27.180128 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1145/2272] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:10:26 2022] PUT /identity/v3/projects/803bbc59642547fb932be578cb70ac3f/users/824b6455aab74659a9fb1e5e8f7c2965/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:10:27.186713 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5676ed-410b-47a0-92e4-5c55d9c51ba7 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:27.188386 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5676ed-410b-47a0-92e4-5c55d9c51ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:27.190116 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5676ed-410b-47a0-92e4-5c55d9c51ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:27.233360 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-ab5676ed-410b-47a0-92e4-5c55d9c51ba7 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:27.235188 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ab5676ed-410b-47a0-92e4-5c55d9c51ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:27.361218 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1128/2273] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:10:27.521574 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b9244-2b17-4d97-af61-29a3389f480a admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:27.525230 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-cc0b9244-2b17-4d97-af61-29a3389f480a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:27.685003 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:27.820615 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b9244-2b17-4d97-af61-29a3389f480a admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:27.838369 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320ce4b-459d-4dba-b197-99817f17d0ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:27.842802 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320ce4b-459d-4dba-b197-99817f17d0ba service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:27.844313 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320ce4b-459d-4dba-b197-99817f17d0ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:27.845296 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320ce4b-459d-4dba-b197-99817f17d0ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:27.876785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9320ce4b-459d-4dba-b197-99817f17d0ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:27.884683 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9320ce4b-459d-4dba-b197-99817f17d0ba service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:27.912315 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1146/2274] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:10:35.037397 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaecbef-ae47-4e9f-9e9f-d36305050bbf None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:35.039000 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaecbef-ae47-4e9f-9e9f-d36305050bbf None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:35.039881 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaecbef-ae47-4e9f-9e9f-d36305050bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:35.206135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-3aaecbef-ae47-4e9f-9e9f-d36305050bbf None None] MFA Rules not processed for user `824b6455aab74659a9fb1e5e8f7c2965`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:35.213967 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3aaecbef-ae47-4e9f-9e9f-d36305050bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:35.514376 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1129/2275] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 479 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:10:35.543432 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320ce4b-459d-4dba-b197-99817f17d0ba service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:35.547943 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9320ce4b-459d-4dba-b197-99817f17d0ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:35.850356 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320ce4b-459d-4dba-b197-99817f17d0ba service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:35.869316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1762f83c-512d-4b2e-9dfa-7d186642d9f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:35.872970 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762f83c-512d-4b2e-9dfa-7d186642d9f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:35.873473 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762f83c-512d-4b2e-9dfa-7d186642d9f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:35.874172 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762f83c-512d-4b2e-9dfa-7d186642d9f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:35.914193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1762f83c-512d-4b2e-9dfa-7d186642d9f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:35.917051 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1762f83c-512d-4b2e-9dfa-7d186642d9f2 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:35.964256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1147/2276] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:10:39.342794 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c118f0-6ea0-4041-a432-6b595c512316 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:39.343813 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c118f0-6ea0-4041-a432-6b595c512316 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:39.344367 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c118f0-6ea0-4041-a432-6b595c512316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:39.382470 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-46c118f0-6ea0-4041-a432-6b595c512316 None None] MFA Rules not processed for user `824b6455aab74659a9fb1e5e8f7c2965`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:39.385135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-46c118f0-6ea0-4041-a432-6b595c512316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:39.410254 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1130/2277] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:10:43.949621 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6867f47-0eab-46fc-a464-4a7625c51332 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:43.950593 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6867f47-0eab-46fc-a464-4a7625c51332 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:43.950593 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6867f47-0eab-46fc-a464-4a7625c51332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:44.062080 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d6867f47-0eab-46fc-a464-4a7625c51332 None None] MFA Rules not processed for user `b4ae8decd5354231bebc7bfda44d3285`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:44.065496 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d6867f47-0eab-46fc-a464-4a7625c51332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:44.529544 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c118f0-6ea0-4041-a432-6b595c512316 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:44.532657 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-46c118f0-6ea0-4041-a432-6b595c512316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:44.551973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1148/2278] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 603 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:10:44.623464 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c118f0-6ea0-4041-a432-6b595c512316 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:44.655453 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0437eef-86e6-454f-8d22-d58d47563851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:44.656942 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0437eef-86e6-454f-8d22-d58d47563851 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:44.667301 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0437eef-86e6-454f-8d22-d58d47563851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:44.667863 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0437eef-86e6-454f-8d22-d58d47563851 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:44.668826 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0437eef-86e6-454f-8d22-d58d47563851 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:44.681490 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0437eef-86e6-454f-8d22-d58d47563851 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:44.805524 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1131/2279] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:10:44 2022] POST /identity/v3/projects => generated 373 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:10:44.813450 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6867f47-0eab-46fc-a464-4a7625c51332 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:44.908229 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6867f47-0eab-46fc-a464-4a7625c51332 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:45.155272 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ed6e2-8453-4e3b-b9d1-49bfdb8354dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:45.158005 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ed6e2-8453-4e3b-b9d1-49bfdb8354dd admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:45.159318 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ed6e2-8453-4e3b-b9d1-49bfdb8354dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:45.161253 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ed6e2-8453-4e3b-b9d1-49bfdb8354dd admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:45.163431 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ed6e2-8453-4e3b-b9d1-49bfdb8354dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:45.169299 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ed6e2-8453-4e3b-b9d1-49bfdb8354dd admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:45.221600 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:45.302420 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0437eef-86e6-454f-8d22-d58d47563851 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:45.318761 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1149/2280] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:10:44 2022] POST /identity/v3/users => generated 293 bytes in 507 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:10:45.333699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0437eef-86e6-454f-8d22-d58d47563851 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:45.343332 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c232e8-3797-4ab5-83d8-dafd0035a735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:45.345253 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c232e8-3797-4ab5-83d8-dafd0035a735 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:45.345768 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c232e8-3797-4ab5-83d8-dafd0035a735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:45.346220 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c232e8-3797-4ab5-83d8-dafd0035a735 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:45.353710 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c232e8-3797-4ab5-83d8-dafd0035a735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:45.357968 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c232e8-3797-4ab5-83d8-dafd0035a735 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:45.401398 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ed6e2-8453-4e3b-b9d1-49bfdb8354dd admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:45.427905 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1132/2281] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:45 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:10:45.439280 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ed6e2-8453-4e3b-b9d1-49bfdb8354dd admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:45.469048 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c18a2cc-ea75-43c9-be30-172562d7fd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:45.469872 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18a2cc-ea75-43c9-be30-172562d7fd5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:45.470234 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18a2cc-ea75-43c9-be30-172562d7fd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:45.470687 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18a2cc-ea75-43c9-be30-172562d7fd5a admin admin] PATH_INFO: `/v3/projects/bfa846b417314968807692fa953e79ad/users/c3bcbc78e809494db785342937dc4971/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:45.543916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c18a2cc-ea75-43c9-be30-172562d7fd5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfa846b417314968807692fa953e79ad, user_id=c3bcbc78e809494db785342937dc4971, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:45.548953 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c18a2cc-ea75-43c9-be30-172562d7fd5a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:45.609625 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1150/2282] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:10:45 2022] PUT /identity/v3/projects/bfa846b417314968807692fa953e79ad/users/c3bcbc78e809494db785342937dc4971/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:45.611235 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c232e8-3797-4ab5-83d8-dafd0035a735 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:45.647772 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c232e8-3797-4ab5-83d8-dafd0035a735 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:45.672549 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:45.843350 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8ef813-3519-4019-873d-0e7d745af942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:45.850159 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ef813-3519-4019-873d-0e7d745af942 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:45.858421 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ef813-3519-4019-873d-0e7d745af942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:45.860135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ef813-3519-4019-873d-0e7d745af942 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:45.861720 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8ef813-3519-4019-873d-0e7d745af942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:45.865187 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8ef813-3519-4019-873d-0e7d745af942 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:45.910961 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1133/2283] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:45 2022] GET /identity/v3/roles => generated 1420 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:10:45.912240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c18a2cc-ea75-43c9-be30-172562d7fd5a admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:45.929632 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c18a2cc-ea75-43c9-be30-172562d7fd5a admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:45.950977 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243abe9-22cb-41ac-967a-734961b7497a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:45.956680 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243abe9-22cb-41ac-967a-734961b7497a admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:45.957279 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243abe9-22cb-41ac-967a-734961b7497a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:45.957682 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243abe9-22cb-41ac-967a-734961b7497a admin admin] PATH_INFO: `/v3/projects/bfa846b417314968807692fa953e79ad/users/c3bcbc78e809494db785342937dc4971/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:45.981024 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243abe9-22cb-41ac-967a-734961b7497a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfa846b417314968807692fa953e79ad, user_id=c3bcbc78e809494db785342937dc4971, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:45.984587 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243abe9-22cb-41ac-967a-734961b7497a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:46.062404 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1151/2284] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:10:45 2022] PUT /identity/v3/projects/bfa846b417314968807692fa953e79ad/users/c3bcbc78e809494db785342937dc4971/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:10:46.777616 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8ef813-3519-4019-873d-0e7d745af942 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:46.781794 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-8c8ef813-3519-4019-873d-0e7d745af942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:46.882433 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8ef813-3519-4019-873d-0e7d745af942 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:47.082330 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0d231-19b9-4c27-b728-895072e01dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:47.085735 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0d231-19b9-4c27-b728-895072e01dbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:47.088896 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0d231-19b9-4c27-b728-895072e01dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:47.090309 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0d231-19b9-4c27-b728-895072e01dbe admin admin] PATH_INFO: `/v3/users/58140e8aa8e9496bb41f962dab03227a` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:47.096938 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c0d231-19b9-4c27-b728-895072e01dbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=58140e8aa8e9496bb41f962dab03227a)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:47.117782 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c0d231-19b9-4c27-b728-895072e01dbe admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:47.296264 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-41c0d231-19b9-4c27-b728-895072e01dbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58140e8aa8e9496bb41f962dab03227a'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:47.300325 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-41c0d231-19b9-4c27-b728-895072e01dbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58140e8aa8e9496bb41f962dab03227a'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:47.338053 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-41c0d231-19b9-4c27-b728-895072e01dbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58140e8aa8e9496bb41f962dab03227a'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:47.359106 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-41c0d231-19b9-4c27-b728-895072e01dbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58140e8aa8e9496bb41f962dab03227a'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:47.389112 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-41c0d231-19b9-4c27-b728-895072e01dbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58140e8aa8e9496bb41f962dab03227a'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:47.417931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1134/2285] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:10:46 2022] DELETE /identity/v3/users/58140e8aa8e9496bb41f962dab03227a => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:47.425273 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193b53d-f4c8-4f68-a92d-986ecccc36ca None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:47.426938 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193b53d-f4c8-4f68-a92d-986ecccc36ca None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:47.427653 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193b53d-f4c8-4f68-a92d-986ecccc36ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:47.452193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585af2a-53bc-4bc1-8e0b-b9a9c1493a6c None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:47.453688 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585af2a-53bc-4bc1-8e0b-b9a9c1493a6c None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:47.458811 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585af2a-53bc-4bc1-8e0b-b9a9c1493a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:47.583291 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c193b53d-f4c8-4f68-a92d-986ecccc36ca None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:47.587450 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c193b53d-f4c8-4f68-a92d-986ecccc36ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:47.650259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-7585af2a-53bc-4bc1-8e0b-b9a9c1493a6c None None] MFA Rules not processed for user `8e30910692b4474092f1585a671e2f41`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:47.652308 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7585af2a-53bc-4bc1-8e0b-b9a9c1493a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:47.683213 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:47.784506 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1152/2286] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:47.801731 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193b53d-f4c8-4f68-a92d-986ecccc36ca None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:47.806148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c193b53d-f4c8-4f68-a92d-986ecccc36ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:47.847655 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1135/2287] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:47.849108 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7585af2a-53bc-4bc1-8e0b-b9a9c1493a6c None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:47.850557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7585af2a-53bc-4bc1-8e0b-b9a9c1493a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:47.887227 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7585af2a-53bc-4bc1-8e0b-b9a9c1493a6c None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:47.901065 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e150b6b-0345-4895-a133-1631ce8c7237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:47.902895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e150b6b-0345-4895-a133-1631ce8c7237 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:47.903185 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e150b6b-0345-4895-a133-1631ce8c7237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:47.903475 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e150b6b-0345-4895-a133-1631ce8c7237 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:47.905218 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e150b6b-0345-4895-a133-1631ce8c7237 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:47.909551 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e150b6b-0345-4895-a133-1631ce8c7237 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:48.005842 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1136/2288] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:10:47 2022] POST /identity/v3/projects => generated 379 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:10:48.007180 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e150b6b-0345-4895-a133-1631ce8c7237 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:48.091136 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193b53d-f4c8-4f68-a92d-986ecccc36ca None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:48.176045 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e150b6b-0345-4895-a133-1631ce8c7237 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:48.187580 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6031ff4-0590-44f0-ad8b-f9ea5f548211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:48.189578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6031ff4-0590-44f0-ad8b-f9ea5f548211 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:48.190561 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6031ff4-0590-44f0-ad8b-f9ea5f548211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:48.191393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6031ff4-0590-44f0-ad8b-f9ea5f548211 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:48.192671 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6031ff4-0590-44f0-ad8b-f9ea5f548211 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:48.197941 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6031ff4-0590-44f0-ad8b-f9ea5f548211 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:48.248033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1137/2289] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:10:48 2022] POST /identity/v3/users => generated 296 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:10:48.249368 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6031ff4-0590-44f0-ad8b-f9ea5f548211 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:48.254396 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bcbd3d-880b-4e2d-a8e2-2e294b898615 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:48.257322 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bcbd3d-880b-4e2d-a8e2-2e294b898615 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:48.259316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bcbd3d-880b-4e2d-a8e2-2e294b898615 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:48.261578 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bcbd3d-880b-4e2d-a8e2-2e294b898615 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:48.265610 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6031ff4-0590-44f0-ad8b-f9ea5f548211 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:48.273687 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071bbf26-53ee-463b-8b82-e17ac47c25c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:48.274466 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071bbf26-53ee-463b-8b82-e17ac47c25c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:48.274760 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071bbf26-53ee-463b-8b82-e17ac47c25c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:48.275089 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071bbf26-53ee-463b-8b82-e17ac47c25c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:48.276144 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071bbf26-53ee-463b-8b82-e17ac47c25c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:48.278583 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071bbf26-53ee-463b-8b82-e17ac47c25c1 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:48.290756 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bcbd3d-880b-4e2d-a8e2-2e294b898615 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:48.294617 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bcbd3d-880b-4e2d-a8e2-2e294b898615 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:48.299440 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1138/2290] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:48 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:10:48.301340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071bbf26-53ee-463b-8b82-e17ac47c25c1 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:48.315356 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071bbf26-53ee-463b-8b82-e17ac47c25c1 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:48.326950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740324f-7f98-494d-b07a-9c29b8af4a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:48.327705 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1153/2291] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:10:48.328622 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740324f-7f98-494d-b07a-9c29b8af4a15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:48.329020 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740324f-7f98-494d-b07a-9c29b8af4a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:48.331691 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740324f-7f98-494d-b07a-9c29b8af4a15 admin admin] PATH_INFO: `/v3/projects/fb0baf0e988143afb181a1bf178a1c53/users/612318edeb824e3abd82a00fc7fdc8df/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:48.378303 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b740324f-7f98-494d-b07a-9c29b8af4a15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb0baf0e988143afb181a1bf178a1c53, user_id=612318edeb824e3abd82a00fc7fdc8df, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:48.397636 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b740324f-7f98-494d-b07a-9c29b8af4a15 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:48.459198 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1139/2292] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:10:48 2022] PUT /identity/v3/projects/fb0baf0e988143afb181a1bf178a1c53/users/612318edeb824e3abd82a00fc7fdc8df/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:48.463658 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bcbd3d-880b-4e2d-a8e2-2e294b898615 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:48.495623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bcbd3d-880b-4e2d-a8e2-2e294b898615 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:48.564697 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:48.840808 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f82d5-a0f0-4b39-beed-006b0dd7b537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:48.841464 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f82d5-a0f0-4b39-beed-006b0dd7b537 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:48.841743 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f82d5-a0f0-4b39-beed-006b0dd7b537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:48.842176 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f82d5-a0f0-4b39-beed-006b0dd7b537 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:48.842910 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f82d5-a0f0-4b39-beed-006b0dd7b537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:48.845773 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f82d5-a0f0-4b39-beed-006b0dd7b537 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:48.876319 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1154/2293] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:48 2022] GET /identity/v3/roles => generated 1420 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:10:48.881399 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740324f-7f98-494d-b07a-9c29b8af4a15 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:48.895954 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740324f-7f98-494d-b07a-9c29b8af4a15 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:48.902444 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9329319a-2c49-46c6-9f53-2e7f96daa0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:48.903588 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329319a-2c49-46c6-9f53-2e7f96daa0c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:48.904004 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329319a-2c49-46c6-9f53-2e7f96daa0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:48.904310 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329319a-2c49-46c6-9f53-2e7f96daa0c2 admin admin] PATH_INFO: `/v3/projects/fb0baf0e988143afb181a1bf178a1c53/users/612318edeb824e3abd82a00fc7fdc8df/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:48.907480 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9329319a-2c49-46c6-9f53-2e7f96daa0c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb0baf0e988143afb181a1bf178a1c53, user_id=612318edeb824e3abd82a00fc7fdc8df, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:48.910675 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9329319a-2c49-46c6-9f53-2e7f96daa0c2 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:48.931600 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1140/2294] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:10:48 2022] PUT /identity/v3/projects/fb0baf0e988143afb181a1bf178a1c53/users/612318edeb824e3abd82a00fc7fdc8df/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:52.379517 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f82d5-a0f0-4b39-beed-006b0dd7b537 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:52.382893 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-087f82d5-a0f0-4b39-beed-006b0dd7b537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:52.439209 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f82d5-a0f0-4b39-beed-006b0dd7b537 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:52.725300 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d20a4f-40c8-4a00-bc3b-0cbf06cdb498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:52.726350 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d20a4f-40c8-4a00-bc3b-0cbf06cdb498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:52.726779 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d20a4f-40c8-4a00-bc3b-0cbf06cdb498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:52.727490 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d20a4f-40c8-4a00-bc3b-0cbf06cdb498 admin admin] PATH_INFO: `/v3/projects/fb3f97da433d4d83b8c2837575ab11f1` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:52.729742 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d20a4f-40c8-4a00-bc3b-0cbf06cdb498 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb3f97da433d4d83b8c2837575ab11f1)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:52.740556 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d20a4f-40c8-4a00-bc3b-0cbf06cdb498 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:52.948711 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-92d20a4f-40c8-4a00-bc3b-0cbf06cdb498 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb3f97da433d4d83b8c2837575ab11f1', 'request_id': 'req-92d20a4f-40c8-4a00-bc3b-0cbf06cdb498'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:52.974689 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-92d20a4f-40c8-4a00-bc3b-0cbf06cdb498 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb3f97da433d4d83b8c2837575ab11f1', 'request_id': 'req-92d20a4f-40c8-4a00-bc3b-0cbf06cdb498'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:53.004121 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-92d20a4f-40c8-4a00-bc3b-0cbf06cdb498 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb3f97da433d4d83b8c2837575ab11f1', 'request_id': 'req-92d20a4f-40c8-4a00-bc3b-0cbf06cdb498'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:53.057085 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-92d20a4f-40c8-4a00-bc3b-0cbf06cdb498 admin admin] The token cache is being invalidate because project fb3f97da433d4d83b8c2837575ab11f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:10:53.061682 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-92d20a4f-40c8-4a00-bc3b-0cbf06cdb498 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:53.065115 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-92d20a4f-40c8-4a00-bc3b-0cbf06cdb498 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:53.075049 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1155/2295] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:10:52 2022] DELETE /identity/v3/projects/fb3f97da433d4d83b8c2837575ab11f1 => generated 0 bytes in 692 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:53.076842 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f76e4-99fe-4fea-aeb2-066ba6c703f1 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:53.077400 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f76e4-99fe-4fea-aeb2-066ba6c703f1 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:53.077499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f76e4-99fe-4fea-aeb2-066ba6c703f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:53.137993 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-b41f76e4-99fe-4fea-aeb2-066ba6c703f1 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:53.141156 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-b41f76e4-99fe-4fea-aeb2-066ba6c703f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:53.288087 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:53.605342 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1141/2296] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 515 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:53.605649 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cebd05a-3b33-4944-bf83-ee6b71d1e062 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:53.606537 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cebd05a-3b33-4944-bf83-ee6b71d1e062 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:53.606537 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cebd05a-3b33-4944-bf83-ee6b71d1e062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:53.647640 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-9cebd05a-3b33-4944-bf83-ee6b71d1e062 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:53.650942 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9cebd05a-3b33-4944-bf83-ee6b71d1e062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:53.760641 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f76e4-99fe-4fea-aeb2-066ba6c703f1 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:53.763024 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1156/2297] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:53.825567 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f76e4-99fe-4fea-aeb2-066ba6c703f1 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:53.845510 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f498af59-4662-49f8-aeb5-a5bb122939c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:53.849441 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498af59-4662-49f8-aeb5-a5bb122939c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:53.849925 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498af59-4662-49f8-aeb5-a5bb122939c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:53.850419 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498af59-4662-49f8-aeb5-a5bb122939c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:53.855244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f498af59-4662-49f8-aeb5-a5bb122939c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:10:53.855838 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f498af59-4662-49f8-aeb5-a5bb122939c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:53.866825 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f498af59-4662-49f8-aeb5-a5bb122939c3 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:53.939717 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1142/2298] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:10:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:10:53.940929 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cebd05a-3b33-4944-bf83-ee6b71d1e062 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:53.975343 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cebd05a-3b33-4944-bf83-ee6b71d1e062 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:54.015847 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef10259-375d-453e-be92-57d267cfbb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:54.022022 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef10259-375d-453e-be92-57d267cfbb0f admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:54.023263 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef10259-375d-453e-be92-57d267cfbb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:54.026118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef10259-375d-453e-be92-57d267cfbb0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:54.027628 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef10259-375d-453e-be92-57d267cfbb0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:54.040310 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef10259-375d-453e-be92-57d267cfbb0f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:54.164693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1157/2299] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:10:53 2022] POST /identity/v3/projects => generated 385 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:10:54.170238 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f498af59-4662-49f8-aeb5-a5bb122939c3 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:54.200344 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f498af59-4662-49f8-aeb5-a5bb122939c3 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:54.468740 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8bb0a2-7101-426f-8963-90e39150d99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:54.469784 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8bb0a2-7101-426f-8963-90e39150d99f admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:54.470260 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8bb0a2-7101-426f-8963-90e39150d99f admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:54.470710 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8bb0a2-7101-426f-8963-90e39150d99f admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:54.471897 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8bb0a2-7101-426f-8963-90e39150d99f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:54.478208 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8bb0a2-7101-426f-8963-90e39150d99f admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:54.541105 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1143/2300] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:10:54 2022] POST /identity/v3/users => generated 293 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:10:54.542405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef10259-375d-453e-be92-57d267cfbb0f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:54.575340 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef10259-375d-453e-be92-57d267cfbb0f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:54.610645 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff759dc9-dd15-4fe5-8d3e-fe4f24c3a212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:54.611687 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff759dc9-dd15-4fe5-8d3e-fe4f24c3a212 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:54.641211 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff759dc9-dd15-4fe5-8d3e-fe4f24c3a212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:54.641696 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff759dc9-dd15-4fe5-8d3e-fe4f24c3a212 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:54.642936 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff759dc9-dd15-4fe5-8d3e-fe4f24c3a212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:54.659859 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff759dc9-dd15-4fe5-8d3e-fe4f24c3a212 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:54.725503 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1158/2301] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:54 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:10:54.731315 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8bb0a2-7101-426f-8963-90e39150d99f admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:54.765025 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8bb0a2-7101-426f-8963-90e39150d99f admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:54.787185 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b91bc-97d2-4c2e-b5de-c573bebcd61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:54.790338 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b91bc-97d2-4c2e-b5de-c573bebcd61a admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:54.794822 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b91bc-97d2-4c2e-b5de-c573bebcd61a admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:54.795523 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b91bc-97d2-4c2e-b5de-c573bebcd61a admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:54.800395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b91bc-97d2-4c2e-b5de-c573bebcd61a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:54.812030 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b91bc-97d2-4c2e-b5de-c573bebcd61a admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:54.868578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1144/2302] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:54 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:10:54.868976 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff759dc9-dd15-4fe5-8d3e-fe4f24c3a212 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:54.877193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:54.885835 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff759dc9-dd15-4fe5-8d3e-fe4f24c3a212 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:54.901398 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31867b4-324f-407a-9a89-30e68428cee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:54.905370 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31867b4-324f-407a-9a89-30e68428cee1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:54.906826 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31867b4-324f-407a-9a89-30e68428cee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:54.909013 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31867b4-324f-407a-9a89-30e68428cee1 admin admin] PATH_INFO: `/v3/projects/cceb4402f3334625bcca351aa005a575/users/84f2a47778d34211b4ddfaeb3fabc474/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:54.975753 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31867b4-324f-407a-9a89-30e68428cee1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cceb4402f3334625bcca351aa005a575, user_id=84f2a47778d34211b4ddfaeb3fabc474, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:54.991261 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31867b4-324f-407a-9a89-30e68428cee1 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:55.126696 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1159/2303] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:10:54 2022] PUT /identity/v3/projects/cceb4402f3334625bcca351aa005a575/users/84f2a47778d34211b4ddfaeb3fabc474/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:55.149075 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab13dcf1-1794-4d79-b5de-6cde79467b25 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:55.149365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab13dcf1-1794-4d79-b5de-6cde79467b25 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:55.149644 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab13dcf1-1794-4d79-b5de-6cde79467b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:55.251382 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-ab13dcf1-1794-4d79-b5de-6cde79467b25 None None] MFA Rules not processed for user `84f2a47778d34211b4ddfaeb3fabc474`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:55.253541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ab13dcf1-1794-4d79-b5de-6cde79467b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:55.497959 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1145/2304] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:55.556121 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31867b4-324f-407a-9a89-30e68428cee1 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:55.558883 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d31867b4-324f-407a-9a89-30e68428cee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:55.816908 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31867b4-324f-407a-9a89-30e68428cee1 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:55.831055 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad134b-d85f-49f3-b0ca-99d2188e88c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:55.850316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad134b-d85f-49f3-b0ca-99d2188e88c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:55.851592 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad134b-d85f-49f3-b0ca-99d2188e88c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:55.852387 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad134b-d85f-49f3-b0ca-99d2188e88c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:55.880383 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad134b-d85f-49f3-b0ca-99d2188e88c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:55.890509 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad134b-d85f-49f3-b0ca-99d2188e88c1 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:55.912054 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1160/2305] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:10:59.486673 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bef291-27eb-4d46-bb01-0ef51fd28eb9 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:59.487703 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bef291-27eb-4d46-bb01-0ef51fd28eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:59.487703 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bef291-27eb-4d46-bb01-0ef51fd28eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:59.519246 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-73bef291-27eb-4d46-bb01-0ef51fd28eb9 None None] MFA Rules not processed for user `ec40d75e772442d2a97e9bf18b4b90c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:59.528920 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-73bef291-27eb-4d46-bb01-0ef51fd28eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:59.578029 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:59.703704 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1146/2306] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:10:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:11:00.212103 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad134b-d85f-49f3-b0ca-99d2188e88c1 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:00.360907 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad134b-d85f-49f3-b0ca-99d2188e88c1 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:00.367619 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869ca8a-02ed-402f-b0d3-935f81e4573d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:00.368401 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869ca8a-02ed-402f-b0d3-935f81e4573d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:00.368699 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869ca8a-02ed-402f-b0d3-935f81e4573d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:00.369158 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869ca8a-02ed-402f-b0d3-935f81e4573d admin admin] PATH_INFO: `/v3/users/84f2a47778d34211b4ddfaeb3fabc474` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:00.370874 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869ca8a-02ed-402f-b0d3-935f81e4573d admin admin] RBAC: Authorizing `identity:delete_user(user_id=84f2a47778d34211b4ddfaeb3fabc474)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:00.374610 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869ca8a-02ed-402f-b0d3-935f81e4573d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:00.478048 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b869ca8a-02ed-402f-b0d3-935f81e4573d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84f2a47778d34211b4ddfaeb3fabc474'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:11:00.478780 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b869ca8a-02ed-402f-b0d3-935f81e4573d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84f2a47778d34211b4ddfaeb3fabc474'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:11:00.513550 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b869ca8a-02ed-402f-b0d3-935f81e4573d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84f2a47778d34211b4ddfaeb3fabc474'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:11:00.537259 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b869ca8a-02ed-402f-b0d3-935f81e4573d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84f2a47778d34211b4ddfaeb3fabc474'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:11:00.539480 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b869ca8a-02ed-402f-b0d3-935f81e4573d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84f2a47778d34211b4ddfaeb3fabc474'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:11:00.561656 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:00.573746 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1161/2307] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:11:00 2022] DELETE /identity/v3/users/84f2a47778d34211b4ddfaeb3fabc474 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:11:00.579668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd8b81-f60f-4e41-8498-62cc19032567 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:00.580753 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd8b81-f60f-4e41-8498-62cc19032567 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:00.580753 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd8b81-f60f-4e41-8498-62cc19032567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:00.649639 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-7fbd8b81-f60f-4e41-8498-62cc19032567 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:00.652605 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7fbd8b81-f60f-4e41-8498-62cc19032567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:00.773248 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1147/2308] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:11:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:11:00.825184 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869ca8a-02ed-402f-b0d3-935f81e4573d admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:00.827052 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b869ca8a-02ed-402f-b0d3-935f81e4573d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:01.049895 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869ca8a-02ed-402f-b0d3-935f81e4573d admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:01.059452 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1447aa-4847-45ba-ba95-7f6c53c9e226 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:01.061189 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1447aa-4847-45ba-ba95-7f6c53c9e226 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:01.061942 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1447aa-4847-45ba-ba95-7f6c53c9e226 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:01.062919 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1447aa-4847-45ba-ba95-7f6c53c9e226 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:01.085005 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1447aa-4847-45ba-ba95-7f6c53c9e226 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:01.088732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1447aa-4847-45ba-ba95-7f6c53c9e226 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:01.108217 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1162/2309] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:11:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:11:01.301826 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffead5-488d-4a27-99e7-588df5e3ccd2 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:01.302357 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffead5-488d-4a27-99e7-588df5e3ccd2 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:01.302444 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffead5-488d-4a27-99e7-588df5e3ccd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:01.341989 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-fbffead5-488d-4a27-99e7-588df5e3ccd2 None None] MFA Rules not processed for user `a3e93dfa85244133b1641a431994b47f`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:01.345625 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-fbffead5-488d-4a27-99e7-588df5e3ccd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:01.515942 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1148/2310] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:11:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:11:02.703483 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1447aa-4847-45ba-ba95-7f6c53c9e226 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:02.705309 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9d1447aa-4847-45ba-ba95-7f6c53c9e226 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:02.746666 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1447aa-4847-45ba-ba95-7f6c53c9e226 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:02.766506 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa945550-b593-46e5-aa59-7899f579e09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:02.767034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa945550-b593-46e5-aa59-7899f579e09c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:02.767386 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa945550-b593-46e5-aa59-7899f579e09c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:02.767894 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa945550-b593-46e5-aa59-7899f579e09c admin admin] PATH_INFO: `/v3/projects/cceb4402f3334625bcca351aa005a575` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:02.769887 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa945550-b593-46e5-aa59-7899f579e09c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cceb4402f3334625bcca351aa005a575)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:02.772238 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa945550-b593-46e5-aa59-7899f579e09c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:02.924235 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fa945550-b593-46e5-aa59-7899f579e09c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cceb4402f3334625bcca351aa005a575', 'request_id': 'req-fa945550-b593-46e5-aa59-7899f579e09c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:11:02.956227 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fa945550-b593-46e5-aa59-7899f579e09c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cceb4402f3334625bcca351aa005a575', 'request_id': 'req-fa945550-b593-46e5-aa59-7899f579e09c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:11:02.993756 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fa945550-b593-46e5-aa59-7899f579e09c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cceb4402f3334625bcca351aa005a575', 'request_id': 'req-fa945550-b593-46e5-aa59-7899f579e09c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:11:03.071299 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fa945550-b593-46e5-aa59-7899f579e09c admin admin] The token cache is being invalidate because project cceb4402f3334625bcca351aa005a575 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:11:03.075546 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fa945550-b593-46e5-aa59-7899f579e09c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:11:03.078325 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fa945550-b593-46e5-aa59-7899f579e09c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:11:03.083370 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1163/2311] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:11:02 2022] DELETE /identity/v3/projects/cceb4402f3334625bcca351aa005a575 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:11:03.091723 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd7b0b-39d2-4063-9584-2b2a49f2836d None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:03.092717 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd7b0b-39d2-4063-9584-2b2a49f2836d None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:03.093029 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd7b0b-39d2-4063-9584-2b2a49f2836d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:03.141150 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-67fd7b0b-39d2-4063-9584-2b2a49f2836d None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:03.144658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-67fd7b0b-39d2-4063-9584-2b2a49f2836d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:03.316717 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1149/2312] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:11:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:11:03.332316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d3507-ec1e-43c9-a8d9-05dd8f087359 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:03.334415 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d3507-ec1e-43c9-a8d9-05dd8f087359 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:03.335134 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d3507-ec1e-43c9-a8d9-05dd8f087359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:03.375192 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-771d3507-ec1e-43c9-a8d9-05dd8f087359 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:03.377599 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-771d3507-ec1e-43c9-a8d9-05dd8f087359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:03.401478 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd7b0b-39d2-4063-9584-2b2a49f2836d None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:03.410540 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1164/2313] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:11:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:11:03.421080 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd7b0b-39d2-4063-9584-2b2a49f2836d None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:03.430977 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809ca45-fdcb-4634-b20b-2e071b8befd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:03.432095 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809ca45-fdcb-4634-b20b-2e071b8befd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:03.432649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809ca45-fdcb-4634-b20b-2e071b8befd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:03.433132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809ca45-fdcb-4634-b20b-2e071b8befd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:03.434431 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c809ca45-fdcb-4634-b20b-2e071b8befd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:11:03.434946 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c809ca45-fdcb-4634-b20b-2e071b8befd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:03.438344 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c809ca45-fdcb-4634-b20b-2e071b8befd0 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:03.469390 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1150/2314] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:11:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:11:03.477751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d3507-ec1e-43c9-a8d9-05dd8f087359 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:03.505026 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d3507-ec1e-43c9-a8d9-05dd8f087359 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:03.530291 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63128f20-41d8-4d7a-9891-22432900c4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:03.531917 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63128f20-41d8-4d7a-9891-22432900c4ce admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:03.532448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63128f20-41d8-4d7a-9891-22432900c4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:03.537410 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63128f20-41d8-4d7a-9891-22432900c4ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:03.537410 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63128f20-41d8-4d7a-9891-22432900c4ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:03.538208 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63128f20-41d8-4d7a-9891-22432900c4ce admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:03.672567 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1165/2315] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:11:03 2022] POST /identity/v3/projects => generated 389 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:11:03.674642 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809ca45-fdcb-4634-b20b-2e071b8befd0 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:03.713215 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:03.733088 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809ca45-fdcb-4634-b20b-2e071b8befd0 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:03.894822 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdeb42c-7b4b-4f6b-b02b-a9a983a365dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:03.897720 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdeb42c-7b4b-4f6b-b02b-a9a983a365dd admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:03.898476 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdeb42c-7b4b-4f6b-b02b-a9a983a365dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:03.899282 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdeb42c-7b4b-4f6b-b02b-a9a983a365dd admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:03.902271 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdeb42c-7b4b-4f6b-b02b-a9a983a365dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:03.908174 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdeb42c-7b4b-4f6b-b02b-a9a983a365dd admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:03.979837 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1151/2316] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:11:03 2022] POST /identity/v3/users => generated 295 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:11:03.985867 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63128f20-41d8-4d7a-9891-22432900c4ce admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:04.027011 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63128f20-41d8-4d7a-9891-22432900c4ce admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:04.046567 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76e882-988b-4e77-92bc-19a211c012e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:04.047427 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76e882-988b-4e77-92bc-19a211c012e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:04.047690 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76e882-988b-4e77-92bc-19a211c012e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:04.047858 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76e882-988b-4e77-92bc-19a211c012e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:04.048877 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae76e882-988b-4e77-92bc-19a211c012e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:04.051229 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae76e882-988b-4e77-92bc-19a211c012e1 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:04.089461 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1166/2317] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:11:03 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:11:04.095975 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdeb42c-7b4b-4f6b-b02b-a9a983a365dd admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:04.120854 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdeb42c-7b4b-4f6b-b02b-a9a983a365dd admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:04.137821 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0b6950-6497-4077-a801-ad4d380e61a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:04.139513 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b6950-6497-4077-a801-ad4d380e61a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:04.140022 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b6950-6497-4077-a801-ad4d380e61a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:04.140632 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b6950-6497-4077-a801-ad4d380e61a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:04.142356 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0b6950-6497-4077-a801-ad4d380e61a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:04.151120 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0b6950-6497-4077-a801-ad4d380e61a3 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:04.174557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1152/2318] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:11:04.180507 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76e882-988b-4e77-92bc-19a211c012e1 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:04.200492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76e882-988b-4e77-92bc-19a211c012e1 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:04.222836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c46b7-c9f4-4b53-aa98-ef42e31084ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:04.226017 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c46b7-c9f4-4b53-aa98-ef42e31084ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:04.226692 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c46b7-c9f4-4b53-aa98-ef42e31084ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:04.227285 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c46b7-c9f4-4b53-aa98-ef42e31084ea admin admin] PATH_INFO: `/v3/projects/e27ec0f3e5d0435c8a1d91ade407fba7/users/612bd329acf64b90a8d9fc86828a18f1/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:04.272519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014c46b7-c9f4-4b53-aa98-ef42e31084ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=e27ec0f3e5d0435c8a1d91ade407fba7, user_id=612bd329acf64b90a8d9fc86828a18f1, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:04.281608 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014c46b7-c9f4-4b53-aa98-ef42e31084ea admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:04.352701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1167/2319] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:11:04 2022] PUT /identity/v3/projects/e27ec0f3e5d0435c8a1d91ade407fba7/users/612bd329acf64b90a8d9fc86828a18f1/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:11:04.504012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bb7da-ec20-4971-b6e5-4b8c3d96cb93 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:04.504340 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bb7da-ec20-4971-b6e5-4b8c3d96cb93 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:04.504748 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bb7da-ec20-4971-b6e5-4b8c3d96cb93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:04.656005 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-648bb7da-ec20-4971-b6e5-4b8c3d96cb93 None None] MFA Rules not processed for user `612bd329acf64b90a8d9fc86828a18f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:04.657935 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-648bb7da-ec20-4971-b6e5-4b8c3d96cb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:04.873726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1153/2320] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:11:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:11:04.895691 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c46b7-c9f4-4b53-aa98-ef42e31084ea admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:04.899776 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-014c46b7-c9f4-4b53-aa98-ef42e31084ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:04.943035 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91957c-9a7d-4276-b660-2c2ed4c1697d None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:04.943314 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91957c-9a7d-4276-b660-2c2ed4c1697d None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:04.943584 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91957c-9a7d-4276-b660-2c2ed4c1697d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:04.967844 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:04.999710 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-5d91957c-9a7d-4276-b660-2c2ed4c1697d None None] MFA Rules not processed for user `e07e83c0bdba47d39c09308eba1bdaae`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:05.001669 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5d91957c-9a7d-4276-b660-2c2ed4c1697d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:05.114027 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c46b7-c9f4-4b53-aa98-ef42e31084ea admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:05.135219 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03dd94-4643-454d-83fa-23d303fb0c9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:05.135949 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03dd94-4643-454d-83fa-23d303fb0c9a service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:05.136235 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03dd94-4643-454d-83fa-23d303fb0c9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:05.138853 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03dd94-4643-454d-83fa-23d303fb0c9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:05.194494 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b03dd94-4643-454d-83fa-23d303fb0c9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:05.197388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b03dd94-4643-454d-83fa-23d303fb0c9a service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:05.227322 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1154/2321] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Aug 1 14:11:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:11:05.252056 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91957c-9a7d-4276-b660-2c2ed4c1697d None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:05.254790 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5d91957c-9a7d-4276-b660-2c2ed4c1697d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:05.269438 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1168/2322] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:11:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:11:05.325954 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:05.489180 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91957c-9a7d-4276-b660-2c2ed4c1697d None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:05.510259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23952c9-ce44-4b94-bc7d-0d6caa5333e9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cda7f365ba0405daaf86870960f12d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:05.510259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23952c9-ce44-4b94-bc7d-0d6caa5333e9 service swift] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:05.510682 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23952c9-ce44-4b94-bc7d-0d6caa5333e9 service swift] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:05.511200 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23952c9-ce44-4b94-bc7d-0d6caa5333e9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:05.530414 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23952c9-ce44-4b94-bc7d-0d6caa5333e9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:05.535121 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23952c9-ce44-4b94-bc7d-0d6caa5333e9 service swift] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:05.574429 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1155/2323] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 14:11:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:11:12.513567 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41906529-3b89-4cd0-99d3-832e8c793ff3 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:12.514101 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41906529-3b89-4cd0-99d3-832e8c793ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:12.514235 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41906529-3b89-4cd0-99d3-832e8c793ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:12.707622 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-41906529-3b89-4cd0-99d3-832e8c793ff3 None None] MFA Rules not processed for user `c3bcbc78e809494db785342937dc4971`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:12.709710 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-41906529-3b89-4cd0-99d3-832e8c793ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:13.179835 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1169/2324] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:11:12 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 668 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:11:13.206960 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23952c9-ce44-4b94-bc7d-0d6caa5333e9 service swift] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:13.212593 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-f23952c9-ce44-4b94-bc7d-0d6caa5333e9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:13.551300 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23952c9-ce44-4b94-bc7d-0d6caa5333e9 service swift] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:13.559462 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff5060-af40-46f5-9d53-6e5344ba50be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:13.561364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff5060-af40-46f5-9d53-6e5344ba50be service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:13.561942 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff5060-af40-46f5-9d53-6e5344ba50be service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:13.562442 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff5060-af40-46f5-9d53-6e5344ba50be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:13.564964 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1eff5060-af40-46f5-9d53-6e5344ba50be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:13.783714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eff5060-af40-46f5-9d53-6e5344ba50be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:13.785734 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eff5060-af40-46f5-9d53-6e5344ba50be service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:13.817876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1156/2325] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:11:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:11:14.207390 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93769c-8f17-4646-8234-c3785cc752eb None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:14.207996 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93769c-8f17-4646-8234-c3785cc752eb None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:14.207996 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93769c-8f17-4646-8234-c3785cc752eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:14.255056 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8b93769c-8f17-4646-8234-c3785cc752eb None None] MFA Rules not processed for user `612318edeb824e3abd82a00fc7fdc8df`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:14.256449 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8b93769c-8f17-4646-8234-c3785cc752eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:14.426708 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1170/2326] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:11:14 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:11:14.463801 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff5060-af40-46f5-9d53-6e5344ba50be service nova] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:14.501119 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff5060-af40-46f5-9d53-6e5344ba50be service nova] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:14.513890 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b6912-d7c2-4122-a068-5d849ea68c9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:14.515272 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b6912-d7c2-4122-a068-5d849ea68c9f service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:14.515832 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b6912-d7c2-4122-a068-5d849ea68c9f service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:14.516515 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b6912-d7c2-4122-a068-5d849ea68c9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:14.520508 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-609b6912-d7c2-4122-a068-5d849ea68c9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:14.624179 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:14.662363 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609b6912-d7c2-4122-a068-5d849ea68c9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:14.665793 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609b6912-d7c2-4122-a068-5d849ea68c9f service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:14.688126 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1157/2327] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:11:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:11:19.664568 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82f4af-760c-4a12-aaac-38cfdd7245dc None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:19.665554 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82f4af-760c-4a12-aaac-38cfdd7245dc None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:19.665554 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82f4af-760c-4a12-aaac-38cfdd7245dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:19.702172 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-5d82f4af-760c-4a12-aaac-38cfdd7245dc None None] MFA Rules not processed for user `b4ae8decd5354231bebc7bfda44d3285`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:19.704871 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5d82f4af-760c-4a12-aaac-38cfdd7245dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:19.720546 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1171/2328] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:11:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:11:24.678078 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=8 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:29.388901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b6912-d7c2-4122-a068-5d849ea68c9f service nova] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:29.473676 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b6912-d7c2-4122-a068-5d849ea68c9f service nova] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:29.480678 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c15cb4-a54b-4362-ae0c-5c56cb79c818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:29.482148 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c15cb4-a54b-4362-ae0c-5c56cb79c818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:29.482993 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c15cb4-a54b-4362-ae0c-5c56cb79c818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:29.483384 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c15cb4-a54b-4362-ae0c-5c56cb79c818 admin admin] PATH_INFO: `/v3/users/612bd329acf64b90a8d9fc86828a18f1` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:29.485137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c15cb4-a54b-4362-ae0c-5c56cb79c818 admin admin] RBAC: Authorizing `identity:delete_user(user_id=612bd329acf64b90a8d9fc86828a18f1)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:29.487569 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c15cb4-a54b-4362-ae0c-5c56cb79c818 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:29.591219 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-c7c15cb4-a54b-4362-ae0c-5c56cb79c818 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '612bd329acf64b90a8d9fc86828a18f1'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:11:29.592570 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-c7c15cb4-a54b-4362-ae0c-5c56cb79c818 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '612bd329acf64b90a8d9fc86828a18f1'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:11:29.616846 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-c7c15cb4-a54b-4362-ae0c-5c56cb79c818 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '612bd329acf64b90a8d9fc86828a18f1'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:11:29.636452 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-c7c15cb4-a54b-4362-ae0c-5c56cb79c818 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '612bd329acf64b90a8d9fc86828a18f1'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:11:29.667214 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-c7c15cb4-a54b-4362-ae0c-5c56cb79c818 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '612bd329acf64b90a8d9fc86828a18f1'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:11:29.678234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1158/2329] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:11:29 2022] DELETE /identity/v3/users/612bd329acf64b90a8d9fc86828a18f1 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:11:29.684760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff020960-6ea7-434f-a94a-5cf701f14e73 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:29.684760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff020960-6ea7-434f-a94a-5cf701f14e73 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:29.684760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff020960-6ea7-434f-a94a-5cf701f14e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:29.731637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-ff020960-6ea7-434f-a94a-5cf701f14e73 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:29.737415 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ff020960-6ea7-434f-a94a-5cf701f14e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:29.794088 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:29.946571 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1172/2330] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:11:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:11:29.968743 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c15cb4-a54b-4362-ae0c-5c56cb79c818 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:29.971851 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c7c15cb4-a54b-4362-ae0c-5c56cb79c818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:30.183040 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c15cb4-a54b-4362-ae0c-5c56cb79c818 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:30.193478 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dffbf8a-34ba-49a8-8516-9a68c013c323 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:30.193939 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dffbf8a-34ba-49a8-8516-9a68c013c323 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:30.195197 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dffbf8a-34ba-49a8-8516-9a68c013c323 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:30.195495 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dffbf8a-34ba-49a8-8516-9a68c013c323 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:30.211771 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dffbf8a-34ba-49a8-8516-9a68c013c323 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:30.214060 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dffbf8a-34ba-49a8-8516-9a68c013c323 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:30.231091 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1159/2331] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:11:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:11:31.691102 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff020960-6ea7-434f-a94a-5cf701f14e73 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:31.692759 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ff020960-6ea7-434f-a94a-5cf701f14e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:31.710443 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff020960-6ea7-434f-a94a-5cf701f14e73 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:31.715634 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe41c47-bebb-472c-be0b-4748d936c6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:31.716457 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe41c47-bebb-472c-be0b-4748d936c6b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:31.716961 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe41c47-bebb-472c-be0b-4748d936c6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:31.717545 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe41c47-bebb-472c-be0b-4748d936c6b4 admin admin] PATH_INFO: `/v3/projects/e27ec0f3e5d0435c8a1d91ade407fba7` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:31.719138 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe41c47-bebb-472c-be0b-4748d936c6b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e27ec0f3e5d0435c8a1d91ade407fba7)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:31.722945 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe41c47-bebb-472c-be0b-4748d936c6b4 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:31.816522 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cfe41c47-bebb-472c-be0b-4748d936c6b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e27ec0f3e5d0435c8a1d91ade407fba7', 'request_id': 'req-cfe41c47-bebb-472c-be0b-4748d936c6b4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:11:31.822767 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cfe41c47-bebb-472c-be0b-4748d936c6b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e27ec0f3e5d0435c8a1d91ade407fba7', 'request_id': 'req-cfe41c47-bebb-472c-be0b-4748d936c6b4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:11:31.833532 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cfe41c47-bebb-472c-be0b-4748d936c6b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e27ec0f3e5d0435c8a1d91ade407fba7', 'request_id': 'req-cfe41c47-bebb-472c-be0b-4748d936c6b4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:11:31.849642 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cfe41c47-bebb-472c-be0b-4748d936c6b4 admin admin] The token cache is being invalidate because project e27ec0f3e5d0435c8a1d91ade407fba7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:11:31.850104 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cfe41c47-bebb-472c-be0b-4748d936c6b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:11:31.851134 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cfe41c47-bebb-472c-be0b-4748d936c6b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:11:31.853450 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1173/2332] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:11:31 2022] DELETE /identity/v3/projects/e27ec0f3e5d0435c8a1d91ade407fba7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:11:33.250565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dffbf8a-34ba-49a8-8516-9a68c013c323 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:33.252117 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-8dffbf8a-34ba-49a8-8516-9a68c013c323 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:33.287190 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:33.359027 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dffbf8a-34ba-49a8-8516-9a68c013c323 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:33.477891 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8729a9d8-b2e1-4f4e-b33a-cae6dab1819c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58380e30789041e4ad9e8652bc628658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:33.478978 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729a9d8-b2e1-4f4e-b33a-cae6dab1819c service placement] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:33.479488 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729a9d8-b2e1-4f4e-b33a-cae6dab1819c service placement] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:33.479982 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729a9d8-b2e1-4f4e-b33a-cae6dab1819c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:33.483206 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-8729a9d8-b2e1-4f4e-b33a-cae6dab1819c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:33.507622 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8729a9d8-b2e1-4f4e-b33a-cae6dab1819c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:33.509694 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8729a9d8-b2e1-4f4e-b33a-cae6dab1819c service placement] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:33.525890 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1160/2333] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:11:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:11:35.350172 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443b70f-cd36-4707-8a3e-3b88da9dcdee None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:35.350172 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443b70f-cd36-4707-8a3e-3b88da9dcdee None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:35.350172 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443b70f-cd36-4707-8a3e-3b88da9dcdee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:35.368902 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-5443b70f-cd36-4707-8a3e-3b88da9dcdee None None] MFA Rules not processed for user `824b6455aab74659a9fb1e5e8f7c2965`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:35.370290 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5443b70f-cd36-4707-8a3e-3b88da9dcdee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:35.452766 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1174/2334] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:11:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:11:43.247324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6efaa7-69d2-49f2-a882-88fe93618368 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:43.248145 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6efaa7-69d2-49f2-a882-88fe93618368 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:43.248145 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6efaa7-69d2-49f2-a882-88fe93618368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:43.289520 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-0f6efaa7-69d2-49f2-a882-88fe93618368 None None] MFA Rules not processed for user `8e30910692b4474092f1585a671e2f41`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:43.291825 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0f6efaa7-69d2-49f2-a882-88fe93618368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:43.389812 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1161/2335] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:11:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:11:45.444472 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=66,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:52.147732 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443b70f-cd36-4707-8a3e-3b88da9dcdee None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:52.150383 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5443b70f-cd36-4707-8a3e-3b88da9dcdee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:52.270655 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443b70f-cd36-4707-8a3e-3b88da9dcdee None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:52.275498 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35466d92-db65-418c-978f-88722e5abd19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:52.276162 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35466d92-db65-418c-978f-88722e5abd19 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:52.276597 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35466d92-db65-418c-978f-88722e5abd19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:52.277030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35466d92-db65-418c-978f-88722e5abd19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:52.279307 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-35466d92-db65-418c-978f-88722e5abd19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:52.362162 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35466d92-db65-418c-978f-88722e5abd19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:52.363633 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35466d92-db65-418c-978f-88722e5abd19 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:52.373851 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1175/2336] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:11:52 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:11:53.382527 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=73 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:02.369417 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=50 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:04.410477 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9d7a5-b0ea-4437-9e47-272325ac1f66 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:04.411341 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9d7a5-b0ea-4437-9e47-272325ac1f66 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:04.411341 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9d7a5-b0ea-4437-9e47-272325ac1f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:04.436128 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-a1f9d7a5-b0ea-4437-9e47-272325ac1f66 None None] MFA Rules not processed for user `e07e83c0bdba47d39c09308eba1bdaae`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:04.437651 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a1f9d7a5-b0ea-4437-9e47-272325ac1f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:04.524497 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1162/2337] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Aug 1 14:12:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:12:04.559707 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35466d92-db65-418c-978f-88722e5abd19 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:04.563370 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-35466d92-db65-418c-978f-88722e5abd19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:04.647737 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35466d92-db65-418c-978f-88722e5abd19 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:04.653229 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c6d38-4a0c-4348-8b36-feb76fbf6c99 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cda7f365ba0405daaf86870960f12d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:04.653866 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c6d38-4a0c-4348-8b36-feb76fbf6c99 service swift] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:04.654148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c6d38-4a0c-4348-8b36-feb76fbf6c99 service swift] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:04.654426 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c6d38-4a0c-4348-8b36-feb76fbf6c99 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:04.664456 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69c6d38-4a0c-4348-8b36-feb76fbf6c99 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:04.666080 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69c6d38-4a0c-4348-8b36-feb76fbf6c99 service swift] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:04.675162 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1176/2338] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 14:12:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 14:12:08.540049 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f9d7a5-b0ea-4437-9e47-272325ac1f66 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:08.543998 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a1f9d7a5-b0ea-4437-9e47-272325ac1f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:08.569248 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f9d7a5-b0ea-4437-9e47-272325ac1f66 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:08.576979 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75c7217-b0d2-490b-b111-b38fe0b8cff8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:08.578272 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c7217-b0d2-490b-b111-b38fe0b8cff8 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:08.578965 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c7217-b0d2-490b-b111-b38fe0b8cff8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:08.579600 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c7217-b0d2-490b-b111-b38fe0b8cff8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:08.581969 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d75c7217-b0d2-490b-b111-b38fe0b8cff8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:08.716812 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75c7217-b0d2-490b-b111-b38fe0b8cff8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:08.719663 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75c7217-b0d2-490b-b111-b38fe0b8cff8 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:08.752016 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1163/2339] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:12:08 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:12:14.671441 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=32 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:18.735202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=55 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:20.668255 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c6d38-4a0c-4348-8b36-feb76fbf6c99 service swift] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:20.697563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c6d38-4a0c-4348-8b36-feb76fbf6c99 service swift] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:20.702824 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42fb67e-a985-4f99-9580-b6474eb4a849 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:20.704100 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fb67e-a985-4f99-9580-b6474eb4a849 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:20.706475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fb67e-a985-4f99-9580-b6474eb4a849 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:20.707126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fb67e-a985-4f99-9580-b6474eb4a849 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:20.714856 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f42fb67e-a985-4f99-9580-b6474eb4a849 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:20.856343 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42fb67e-a985-4f99-9580-b6474eb4a849 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:20.859310 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42fb67e-a985-4f99-9580-b6474eb4a849 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:20.885567 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1177/2340] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:12:20 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:12:22.171215 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19fa42-3048-462a-a5ae-75355e9cc883 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:22.172413 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19fa42-3048-462a-a5ae-75355e9cc883 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:22.172914 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19fa42-3048-462a-a5ae-75355e9cc883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:22.226610 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-8b19fa42-3048-462a-a5ae-75355e9cc883 None None] MFA Rules not processed for user `e07e83c0bdba47d39c09308eba1bdaae`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:22.227902 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-8b19fa42-3048-462a-a5ae-75355e9cc883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:22.258806 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1164/2341] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Aug 1 14:12:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:12:22.329965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42fb67e-a985-4f99-9580-b6474eb4a849 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:22.346942 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42fb67e-a985-4f99-9580-b6474eb4a849 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:22.356001 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5c617-1f6d-4b95-b0cf-5025c61e8eae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cda7f365ba0405daaf86870960f12d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:22.357820 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5c617-1f6d-4b95-b0cf-5025c61e8eae service swift] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:22.358654 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5c617-1f6d-4b95-b0cf-5025c61e8eae service swift] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:22.359249 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5c617-1f6d-4b95-b0cf-5025c61e8eae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:22.370312 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f5c617-1f6d-4b95-b0cf-5025c61e8eae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:22.372511 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f5c617-1f6d-4b95-b0cf-5025c61e8eae service swift] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:22.384116 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1178/2342] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 14:12:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:12:24.284440 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb2bcc-d239-436f-9417-6d4dc8cf0162 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:24.285065 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb2bcc-d239-436f-9417-6d4dc8cf0162 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:24.285065 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb2bcc-d239-436f-9417-6d4dc8cf0162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:24.318379 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-a2cb2bcc-d239-436f-9417-6d4dc8cf0162 None None] MFA Rules not processed for user `e07e83c0bdba47d39c09308eba1bdaae`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:24.319746 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a2cb2bcc-d239-436f-9417-6d4dc8cf0162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:24.340309 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1165/2343] 10.222.0.45 () {58 vars in 967 bytes} [Mon Aug 1 14:12:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:12:24.363586 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5c617-1f6d-4b95-b0cf-5025c61e8eae service swift] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:24.389105 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5c617-1f6d-4b95-b0cf-5025c61e8eae service swift] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:24.404121 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a807f-c5f1-4a5d-bfe5-b4777284e75d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cda7f365ba0405daaf86870960f12d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:24.406080 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a807f-c5f1-4a5d-bfe5-b4777284e75d service swift] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:24.406785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a807f-c5f1-4a5d-bfe5-b4777284e75d service swift] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:24.407465 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a807f-c5f1-4a5d-bfe5-b4777284e75d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:24.429708 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a807f-c5f1-4a5d-bfe5-b4777284e75d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:24.431709 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a807f-c5f1-4a5d-bfe5-b4777284e75d service swift] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:24.449872 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1179/2344] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 14:12:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:12:27.381302 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb2bcc-d239-436f-9417-6d4dc8cf0162 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:27.383273 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a2cb2bcc-d239-436f-9417-6d4dc8cf0162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:27.538855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb2bcc-d239-436f-9417-6d4dc8cf0162 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:27.552527 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb9ba87-32bb-492a-b7a9-8ecaab57bd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:27.554504 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9ba87-32bb-492a-b7a9-8ecaab57bd5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:27.555646 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9ba87-32bb-492a-b7a9-8ecaab57bd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:27.556076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9ba87-32bb-492a-b7a9-8ecaab57bd5b admin admin] PATH_INFO: `/v3/users/824b6455aab74659a9fb1e5e8f7c2965` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:27.557974 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb9ba87-32bb-492a-b7a9-8ecaab57bd5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=824b6455aab74659a9fb1e5e8f7c2965)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:27.562002 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb9ba87-32bb-492a-b7a9-8ecaab57bd5b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:27.637970 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-6bb9ba87-32bb-492a-b7a9-8ecaab57bd5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '824b6455aab74659a9fb1e5e8f7c2965'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:12:27.638834 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-6bb9ba87-32bb-492a-b7a9-8ecaab57bd5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '824b6455aab74659a9fb1e5e8f7c2965'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:12:27.659824 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-6bb9ba87-32bb-492a-b7a9-8ecaab57bd5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '824b6455aab74659a9fb1e5e8f7c2965'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:12:27.682732 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-6bb9ba87-32bb-492a-b7a9-8ecaab57bd5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '824b6455aab74659a9fb1e5e8f7c2965'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:12:27.699174 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-6bb9ba87-32bb-492a-b7a9-8ecaab57bd5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '824b6455aab74659a9fb1e5e8f7c2965'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:12:27.707525 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1166/2345] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:12:27 2022] DELETE /identity/v3/users/824b6455aab74659a9fb1e5e8f7c2965 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:12:27.979049 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a807f-c5f1-4a5d-bfe5-b4777284e75d service swift] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:28.036718 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a807f-c5f1-4a5d-bfe5-b4777284e75d service swift] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:28.152301 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566494d1-de96-4194-be6a-5e8b3ff96880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:28.154156 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566494d1-de96-4194-be6a-5e8b3ff96880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:28.154833 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566494d1-de96-4194-be6a-5e8b3ff96880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:28.155809 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566494d1-de96-4194-be6a-5e8b3ff96880 admin admin] PATH_INFO: `/v3/projects/803bbc59642547fb932be578cb70ac3f` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:28.158890 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566494d1-de96-4194-be6a-5e8b3ff96880 admin admin] RBAC: Authorizing `identity:delete_project(project_id=803bbc59642547fb932be578cb70ac3f)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:28.162239 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566494d1-de96-4194-be6a-5e8b3ff96880 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:28.236580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-566494d1-de96-4194-be6a-5e8b3ff96880 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '803bbc59642547fb932be578cb70ac3f', 'request_id': 'req-566494d1-de96-4194-be6a-5e8b3ff96880'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:12:28.259254 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-566494d1-de96-4194-be6a-5e8b3ff96880 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '803bbc59642547fb932be578cb70ac3f', 'request_id': 'req-566494d1-de96-4194-be6a-5e8b3ff96880'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:12:28.279457 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-566494d1-de96-4194-be6a-5e8b3ff96880 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '803bbc59642547fb932be578cb70ac3f', 'request_id': 'req-566494d1-de96-4194-be6a-5e8b3ff96880'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:12:28.289521 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:28.304384 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-566494d1-de96-4194-be6a-5e8b3ff96880 admin admin] The token cache is being invalidate because project 803bbc59642547fb932be578cb70ac3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:12:28.305203 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-566494d1-de96-4194-be6a-5e8b3ff96880 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:12:28.306101 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-566494d1-de96-4194-be6a-5e8b3ff96880 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:12:28.308654 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1180/2346] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:12:27 2022] DELETE /identity/v3/projects/803bbc59642547fb932be578cb70ac3f => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:12:28.318206 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352081c8-1b91-4c37-a01f-62da380bb63e None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:28.318494 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352081c8-1b91-4c37-a01f-62da380bb63e None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:28.318767 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352081c8-1b91-4c37-a01f-62da380bb63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:28.349304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-352081c8-1b91-4c37-a01f-62da380bb63e None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:28.351401 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-352081c8-1b91-4c37-a01f-62da380bb63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:28.452644 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1167/2347] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:12:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:12:28.470743 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1f6e1-54fc-468b-98c1-13d1708ab8b1 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:28.472013 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1f6e1-54fc-468b-98c1-13d1708ab8b1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:28.473132 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1f6e1-54fc-468b-98c1-13d1708ab8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:28.502300 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-38e1f6e1-54fc-468b-98c1-13d1708ab8b1 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:28.503995 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-38e1f6e1-54fc-468b-98c1-13d1708ab8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:28.519873 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1181/2348] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:12:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:12:28.522150 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352081c8-1b91-4c37-a01f-62da380bb63e None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:28.546159 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352081c8-1b91-4c37-a01f-62da380bb63e None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:28.554590 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06ff62-b51d-40ec-9c6f-5af6c21cbd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:28.555004 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06ff62-b51d-40ec-9c6f-5af6c21cbd64 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:28.555480 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06ff62-b51d-40ec-9c6f-5af6c21cbd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:28.556167 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06ff62-b51d-40ec-9c6f-5af6c21cbd64 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:28.557771 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06ff62-b51d-40ec-9c6f-5af6c21cbd64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:12:28.558202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06ff62-b51d-40ec-9c6f-5af6c21cbd64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:28.561375 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06ff62-b51d-40ec-9c6f-5af6c21cbd64 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:28.564810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:28.596212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1168/2349] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:12:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:12:28.597070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1f6e1-54fc-468b-98c1-13d1708ab8b1 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:28.608945 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1f6e1-54fc-468b-98c1-13d1708ab8b1 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:28.616222 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40f650f-3b95-4b63-aaf9-f28aa66f848b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:28.617005 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f650f-3b95-4b63-aaf9-f28aa66f848b admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:28.617291 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f650f-3b95-4b63-aaf9-f28aa66f848b admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:28.617647 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f650f-3b95-4b63-aaf9-f28aa66f848b admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:28.618375 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40f650f-3b95-4b63-aaf9-f28aa66f848b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:28.620470 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40f650f-3b95-4b63-aaf9-f28aa66f848b admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:28.663910 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1182/2350] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:12:28 2022] POST /identity/v3/projects => generated 371 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:12:28.668935 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06ff62-b51d-40ec-9c6f-5af6c21cbd64 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:28.703479 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06ff62-b51d-40ec-9c6f-5af6c21cbd64 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:28.802249 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f736caa6-d9f3-43f6-9aae-5554f1178aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:28.803299 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736caa6-d9f3-43f6-9aae-5554f1178aed admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:28.804047 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736caa6-d9f3-43f6-9aae-5554f1178aed admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:28.804785 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736caa6-d9f3-43f6-9aae-5554f1178aed admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:28.805756 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f736caa6-d9f3-43f6-9aae-5554f1178aed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:28.808034 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f736caa6-d9f3-43f6-9aae-5554f1178aed admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:28.852111 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1169/2351] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:12:28 2022] POST /identity/v3/users => generated 286 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:12:28.859571 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40f650f-3b95-4b63-aaf9-f28aa66f848b admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:28.877763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40f650f-3b95-4b63-aaf9-f28aa66f848b admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:28.893327 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501da1ad-5c8a-41bf-8e4f-743924972d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:28.893972 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501da1ad-5c8a-41bf-8e4f-743924972d7f admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:28.894304 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501da1ad-5c8a-41bf-8e4f-743924972d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:28.894601 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501da1ad-5c8a-41bf-8e4f-743924972d7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:28.895313 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501da1ad-5c8a-41bf-8e4f-743924972d7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:28.897378 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501da1ad-5c8a-41bf-8e4f-743924972d7f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:28.908236 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1183/2352] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:12:28.912999 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f736caa6-d9f3-43f6-9aae-5554f1178aed admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:28.924803 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f736caa6-d9f3-43f6-9aae-5554f1178aed admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:28.930370 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9fd427-8cfe-4bcb-bb24-8cf9088f0620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:28.932831 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9fd427-8cfe-4bcb-bb24-8cf9088f0620 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:28.933151 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9fd427-8cfe-4bcb-bb24-8cf9088f0620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:28.933486 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9fd427-8cfe-4bcb-bb24-8cf9088f0620 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:28.934261 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9fd427-8cfe-4bcb-bb24-8cf9088f0620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:28.935897 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9fd427-8cfe-4bcb-bb24-8cf9088f0620 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:28.954518 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1170/2353] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:12:28.955848 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501da1ad-5c8a-41bf-8e4f-743924972d7f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:28.967431 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501da1ad-5c8a-41bf-8e4f-743924972d7f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:28.973889 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6cfd7-68a0-4352-bb4a-c58852ce5140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:28.974954 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6cfd7-68a0-4352-bb4a-c58852ce5140 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:28.975242 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6cfd7-68a0-4352-bb4a-c58852ce5140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:28.975520 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6cfd7-68a0-4352-bb4a-c58852ce5140 admin admin] PATH_INFO: `/v3/projects/41acf8074adc499b98b7c74cb9928c4a/users/028ff246ab5a40dab48c5c3c269f1a07/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:28.999257 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6cfd7-68a0-4352-bb4a-c58852ce5140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41acf8074adc499b98b7c74cb9928c4a, user_id=028ff246ab5a40dab48c5c3c269f1a07, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:29.002232 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6cfd7-68a0-4352-bb4a-c58852ce5140 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:29.020336 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1184/2354] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:12:28 2022] PUT /identity/v3/projects/41acf8074adc499b98b7c74cb9928c4a/users/028ff246ab5a40dab48c5c3c269f1a07/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:12:29.033118 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b0842-90a9-4074-a6f4-52d6de281af6 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:29.033410 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b0842-90a9-4074-a6f4-52d6de281af6 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:29.034057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b0842-90a9-4074-a6f4-52d6de281af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:29.067777 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-e18b0842-90a9-4074-a6f4-52d6de281af6 None None] MFA Rules not processed for user `028ff246ab5a40dab48c5c3c269f1a07`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:29.068981 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e18b0842-90a9-4074-a6f4-52d6de281af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:29.172489 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1171/2355] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:12:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:12:29.196330 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6cfd7-68a0-4352-bb4a-c58852ce5140 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:29.202834 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-dcc6cfd7-68a0-4352-bb4a-c58852ce5140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:29.302508 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6cfd7-68a0-4352-bb4a-c58852ce5140 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:29.307207 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2287fbdb-b302-4742-b3f0-2277ad5c5817 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7317c8829cb4fa1b30711f1db575cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:29.308020 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287fbdb-b302-4742-b3f0-2277ad5c5817 service glance] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:29.308498 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287fbdb-b302-4742-b3f0-2277ad5c5817 service glance] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:29.308945 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287fbdb-b302-4742-b3f0-2277ad5c5817 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:29.318483 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2287fbdb-b302-4742-b3f0-2277ad5c5817 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:29.319851 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2287fbdb-b302-4742-b3f0-2277ad5c5817 service glance] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:29.329660 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1185/2356] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:12:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:12:38.169205 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a678f5-5c3d-41b8-817a-0b7b7cef861a None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:38.169205 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a678f5-5c3d-41b8-817a-0b7b7cef861a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:38.169205 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a678f5-5c3d-41b8-817a-0b7b7cef861a None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:38.169901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1172/2357] 10.222.0.45 () {54 vars in 921 bytes} [Mon Aug 1 14:12:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:12:38.176756 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f48034-a90a-4dfc-9295-af64d994c85f None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:38.177338 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f48034-a90a-4dfc-9295-af64d994c85f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:38.177338 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f48034-a90a-4dfc-9295-af64d994c85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:38.201110 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-62f48034-a90a-4dfc-9295-af64d994c85f None None] MFA Rules not processed for user `5b3adc14dcaf4ac79b43bb97fa1ba1d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:38.202470 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-62f48034-a90a-4dfc-9295-af64d994c85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:38.290287 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1186/2358] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Aug 1 14:12:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:12:38.302225 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a678f5-5c3d-41b8-817a-0b7b7cef861a None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:38.304080 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-94a678f5-5c3d-41b8-817a-0b7b7cef861a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:38.319387 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a678f5-5c3d-41b8-817a-0b7b7cef861a None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:38.323600 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590d51f5-9249-4e98-9bfe-fa2ede4568f0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7317c8829cb4fa1b30711f1db575cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:38.324237 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d51f5-9249-4e98-9bfe-fa2ede4568f0 service glance] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:38.324546 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d51f5-9249-4e98-9bfe-fa2ede4568f0 service glance] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:38.324814 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d51f5-9249-4e98-9bfe-fa2ede4568f0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:38.336475 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590d51f5-9249-4e98-9bfe-fa2ede4568f0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:38.337914 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590d51f5-9249-4e98-9bfe-fa2ede4568f0 service glance] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:38.349983 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1173/2359] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:12:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:12:38.432025 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e32ce-15d2-420c-ba02-217ee286ed07 None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:38.432333 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e32ce-15d2-420c-ba02-217ee286ed07 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:38.432632 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e32ce-15d2-420c-ba02-217ee286ed07 None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:38.433990 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1187/2360] 10.222.0.45 () {54 vars in 921 bytes} [Mon Aug 1 14:12:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:12:38.439925 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249802a8-186d-4ced-b699-7c7633aed620 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:38.440255 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249802a8-186d-4ced-b699-7c7633aed620 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:38.440682 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249802a8-186d-4ced-b699-7c7633aed620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:38.464306 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-249802a8-186d-4ced-b699-7c7633aed620 None None] MFA Rules not processed for user `6dd44b7a78594288b6aabb393fd6324a`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:38.465673 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-249802a8-186d-4ced-b699-7c7633aed620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:38.518254 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:38.529460 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1174/2361] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Aug 1 14:12:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:12:38.547673 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e32ce-15d2-420c-ba02-217ee286ed07 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:38.550127 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4d6e32ce-15d2-420c-ba02-217ee286ed07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:38.568493 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e32ce-15d2-420c-ba02-217ee286ed07 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:38.572141 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc5ece0-fd65-40c2-a395-24bc427a2bd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:38.573029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5ece0-fd65-40c2-a395-24bc427a2bd0 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:38.573292 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5ece0-fd65-40c2-a395-24bc427a2bd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:38.573498 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5ece0-fd65-40c2-a395-24bc427a2bd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:38.581987 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc5ece0-fd65-40c2-a395-24bc427a2bd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:38.583483 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc5ece0-fd65-40c2-a395-24bc427a2bd0 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:38.592785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1188/2362] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:12:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:12:38.691790 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf74f2-1fc2-48a2-b597-56410f334182 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:38.692117 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf74f2-1fc2-48a2-b597-56410f334182 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:38.692450 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf74f2-1fc2-48a2-b597-56410f334182 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:38.693827 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1175/2363] 10.222.0.45 () {54 vars in 921 bytes} [Mon Aug 1 14:12:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:12:38.699778 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283b309-6bb7-435f-9576-49fd58332efe None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:38.700019 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283b309-6bb7-435f-9576-49fd58332efe None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:38.700321 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283b309-6bb7-435f-9576-49fd58332efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:38.725034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c283b309-6bb7-435f-9576-49fd58332efe None None] MFA Rules not processed for user `5b3adc14dcaf4ac79b43bb97fa1ba1d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:38.725034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c283b309-6bb7-435f-9576-49fd58332efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:38.734851 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1189/2364] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Aug 1 14:12:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:12:38.746415 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf74f2-1fc2-48a2-b597-56410f334182 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:38.761174 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf74f2-1fc2-48a2-b597-56410f334182 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:38.765740 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2baa82-2651-4e3f-a036-ef279bae95ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:38.766459 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2baa82-2651-4e3f-a036-ef279bae95ac service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:38.766745 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2baa82-2651-4e3f-a036-ef279bae95ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:38.767022 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2baa82-2651-4e3f-a036-ef279bae95ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:38.779433 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2baa82-2651-4e3f-a036-ef279bae95ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:38.781116 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2baa82-2651-4e3f-a036-ef279bae95ac service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:38.793920 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1176/2365] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:12:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:12:41.697752 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3beab-6b5c-4403-af09-f4b9cdd61578 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:41.699034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3beab-6b5c-4403-af09-f4b9cdd61578 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:41.699034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3beab-6b5c-4403-af09-f4b9cdd61578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:41.721492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-08d3beab-6b5c-4403-af09-f4b9cdd61578 None None] MFA Rules not processed for user `e07e83c0bdba47d39c09308eba1bdaae`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:41.722600 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-08d3beab-6b5c-4403-af09-f4b9cdd61578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:41.727311 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:41.807973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1190/2366] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Aug 1 14:12:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:12:41.822679 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2baa82-2651-4e3f-a036-ef279bae95ac service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:41.824543 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-dc2baa82-2651-4e3f-a036-ef279bae95ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:41.932786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2baa82-2651-4e3f-a036-ef279bae95ac service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:41.960462 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb610c-b186-40d1-aadc-9fd411b51642 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cda7f365ba0405daaf86870960f12d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:41.962196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb610c-b186-40d1-aadc-9fd411b51642 service swift] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:41.962587 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb610c-b186-40d1-aadc-9fd411b51642 service swift] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:41.962959 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb610c-b186-40d1-aadc-9fd411b51642 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:41.973209 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbb610c-b186-40d1-aadc-9fd411b51642 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:41.975224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbb610c-b186-40d1-aadc-9fd411b51642 service swift] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:41.985909 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1177/2367] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 14:12:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:12:51.796033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=21 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:51.981444 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=39 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:01.391112 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d3beab-6b5c-4403-af09-f4b9cdd61578 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:01.484171 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d3beab-6b5c-4403-af09-f4b9cdd61578 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:01.492439 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10609664-c8a8-4b8b-a209-407cf5b50e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:01.493579 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10609664-c8a8-4b8b-a209-407cf5b50e0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:01.493579 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10609664-c8a8-4b8b-a209-407cf5b50e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:01.493843 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10609664-c8a8-4b8b-a209-407cf5b50e0f admin admin] PATH_INFO: `/v3/users/028ff246ab5a40dab48c5c3c269f1a07` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:01.494847 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10609664-c8a8-4b8b-a209-407cf5b50e0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=028ff246ab5a40dab48c5c3c269f1a07)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:01.496846 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10609664-c8a8-4b8b-a209-407cf5b50e0f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:01.553754 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-10609664-c8a8-4b8b-a209-407cf5b50e0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '028ff246ab5a40dab48c5c3c269f1a07'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:13:01.554411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-10609664-c8a8-4b8b-a209-407cf5b50e0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '028ff246ab5a40dab48c5c3c269f1a07'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:13:01.570447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-10609664-c8a8-4b8b-a209-407cf5b50e0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '028ff246ab5a40dab48c5c3c269f1a07'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:13:01.578051 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-10609664-c8a8-4b8b-a209-407cf5b50e0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '028ff246ab5a40dab48c5c3c269f1a07'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:13:01.578941 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-10609664-c8a8-4b8b-a209-407cf5b50e0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '028ff246ab5a40dab48c5c3c269f1a07'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:13:01.599580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1191/2368] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:13:01 2022] DELETE /identity/v3/users/028ff246ab5a40dab48c5c3c269f1a07 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:13:01.605420 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6dd2b-a266-4a1f-b160-b359b0d68b9f None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:01.605961 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6dd2b-a266-4a1f-b160-b359b0d68b9f None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:01.605961 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6dd2b-a266-4a1f-b160-b359b0d68b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:01.630376 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-7ef6dd2b-a266-4a1f-b160-b359b0d68b9f None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:01.631493 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7ef6dd2b-a266-4a1f-b160-b359b0d68b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:01.711693 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1178/2369] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:13:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:13:01.722649 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10609664-c8a8-4b8b-a209-407cf5b50e0f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:01.724464 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-10609664-c8a8-4b8b-a209-407cf5b50e0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:01.796009 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10609664-c8a8-4b8b-a209-407cf5b50e0f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:01.800406 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6c2e0-7274-49ea-9c0a-c4107a0bb9c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:01.801175 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6c2e0-7274-49ea-9c0a-c4107a0bb9c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:01.801449 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6c2e0-7274-49ea-9c0a-c4107a0bb9c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:01.801719 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6c2e0-7274-49ea-9c0a-c4107a0bb9c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:01.819517 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b6c2e0-7274-49ea-9c0a-c4107a0bb9c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:01.821063 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b6c2e0-7274-49ea-9c0a-c4107a0bb9c1 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:01.833209 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1192/2370] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:13:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:13:02.237622 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6dd2b-a266-4a1f-b160-b359b0d68b9f None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:02.239233 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7ef6dd2b-a266-4a1f-b160-b359b0d68b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:02.257412 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6dd2b-a266-4a1f-b160-b359b0d68b9f None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:02.263476 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247bfd4c-ec11-4448-9b56-770185c634fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:02.264686 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247bfd4c-ec11-4448-9b56-770185c634fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:02.265067 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247bfd4c-ec11-4448-9b56-770185c634fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:02.265215 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247bfd4c-ec11-4448-9b56-770185c634fa admin admin] PATH_INFO: `/v3/projects/41acf8074adc499b98b7c74cb9928c4a` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:02.266755 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247bfd4c-ec11-4448-9b56-770185c634fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=41acf8074adc499b98b7c74cb9928c4a)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:02.268563 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247bfd4c-ec11-4448-9b56-770185c634fa admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:02.332847 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-247bfd4c-ec11-4448-9b56-770185c634fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41acf8074adc499b98b7c74cb9928c4a', 'request_id': 'req-247bfd4c-ec11-4448-9b56-770185c634fa'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:13:02.339712 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-247bfd4c-ec11-4448-9b56-770185c634fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41acf8074adc499b98b7c74cb9928c4a', 'request_id': 'req-247bfd4c-ec11-4448-9b56-770185c634fa'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:13:02.351740 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-247bfd4c-ec11-4448-9b56-770185c634fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41acf8074adc499b98b7c74cb9928c4a', 'request_id': 'req-247bfd4c-ec11-4448-9b56-770185c634fa'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:13:02.365043 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-247bfd4c-ec11-4448-9b56-770185c634fa admin admin] The token cache is being invalidate because project 41acf8074adc499b98b7c74cb9928c4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:13:02.365370 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-247bfd4c-ec11-4448-9b56-770185c634fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:13:02.366171 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-247bfd4c-ec11-4448-9b56-770185c634fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:13:02.367776 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1179/2371] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:13:02 2022] DELETE /identity/v3/projects/41acf8074adc499b98b7c74cb9928c4a => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:13:02.373266 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb730ce8-1c36-4535-86fa-19c3fc313817 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:02.373619 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb730ce8-1c36-4535-86fa-19c3fc313817 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:02.374033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb730ce8-1c36-4535-86fa-19c3fc313817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:02.395548 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-fb730ce8-1c36-4535-86fa-19c3fc313817 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:02.397085 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fb730ce8-1c36-4535-86fa-19c3fc313817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:02.419730 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:02.480451 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1193/2372] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:13:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:13:02.488948 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c733b-8b42-4a9c-a5a1-58ffcdef7ace None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:02.489132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c733b-8b42-4a9c-a5a1-58ffcdef7ace None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:02.489324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c733b-8b42-4a9c-a5a1-58ffcdef7ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:02.520370 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-ac0c733b-8b42-4a9c-a5a1-58ffcdef7ace None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:02.521881 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ac0c733b-8b42-4a9c-a5a1-58ffcdef7ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:02.530668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1180/2373] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:13:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:13:02.535638 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb730ce8-1c36-4535-86fa-19c3fc313817 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:02.546444 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb730ce8-1c36-4535-86fa-19c3fc313817 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:02.551925 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f3b80-6f81-40bc-8810-e533257294e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:02.552611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f3b80-6f81-40bc-8810-e533257294e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:02.552924 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f3b80-6f81-40bc-8810-e533257294e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:02.553154 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f3b80-6f81-40bc-8810-e533257294e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:02.553865 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f3b80-6f81-40bc-8810-e533257294e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:13:02.554167 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f3b80-6f81-40bc-8810-e533257294e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:02.555798 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f3b80-6f81-40bc-8810-e533257294e1 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:02.572551 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1194/2374] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:13:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:13:02.577742 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c733b-8b42-4a9c-a5a1-58ffcdef7ace None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:02.591541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c733b-8b42-4a9c-a5a1-58ffcdef7ace None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:02.597492 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e5d81-8a60-4d41-8c2a-817240d4a288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:02.598216 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e5d81-8a60-4d41-8c2a-817240d4a288 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:02.598531 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e5d81-8a60-4d41-8c2a-817240d4a288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:02.598804 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e5d81-8a60-4d41-8c2a-817240d4a288 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:02.599490 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e5d81-8a60-4d41-8c2a-817240d4a288 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:02.601496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e5d81-8a60-4d41-8c2a-817240d4a288 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:02.640003 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1181/2375] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:13:02 2022] POST /identity/v3/projects => generated 373 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:13:02.644735 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f3b80-6f81-40bc-8810-e533257294e1 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:02.655854 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f3b80-6f81-40bc-8810-e533257294e1 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:02.725012 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f3583-c58e-412a-bc40-117c505bd5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:02.726074 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f3583-c58e-412a-bc40-117c505bd5db admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:02.726576 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f3583-c58e-412a-bc40-117c505bd5db admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:02.726936 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f3583-c58e-412a-bc40-117c505bd5db admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:02.728086 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f3583-c58e-412a-bc40-117c505bd5db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:02.731906 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f3583-c58e-412a-bc40-117c505bd5db admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:02.765598 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1195/2376] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:13:02 2022] POST /identity/v3/users => generated 287 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:13:02.772374 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e5d81-8a60-4d41-8c2a-817240d4a288 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:02.785463 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e5d81-8a60-4d41-8c2a-817240d4a288 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:02.797067 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d7003-02b8-448d-82e8-dd793ae9a5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:02.797717 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d7003-02b8-448d-82e8-dd793ae9a5f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:02.797998 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d7003-02b8-448d-82e8-dd793ae9a5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:02.798268 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d7003-02b8-448d-82e8-dd793ae9a5f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:02.798972 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d7003-02b8-448d-82e8-dd793ae9a5f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:02.800615 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d7003-02b8-448d-82e8-dd793ae9a5f4 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:02.812291 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1182/2377] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:13:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:13:02.816765 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f3583-c58e-412a-bc40-117c505bd5db admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:02.829759 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f3583-c58e-412a-bc40-117c505bd5db admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:02.835949 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b779115-0e7a-4785-859a-4346c03a1aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:02.836623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b779115-0e7a-4785-859a-4346c03a1aef admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:02.836930 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b779115-0e7a-4785-859a-4346c03a1aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:02.837258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b779115-0e7a-4785-859a-4346c03a1aef admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:02.837888 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b779115-0e7a-4785-859a-4346c03a1aef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:02.839518 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b779115-0e7a-4785-859a-4346c03a1aef admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:02.852886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1196/2378] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:13:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:13:02.858111 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d7003-02b8-448d-82e8-dd793ae9a5f4 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:02.870328 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d7003-02b8-448d-82e8-dd793ae9a5f4 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:02.877016 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b772a0-b789-4104-9dfa-6105e5b16c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:02.877805 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b772a0-b789-4104-9dfa-6105e5b16c8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:02.878086 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b772a0-b789-4104-9dfa-6105e5b16c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:02.878370 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b772a0-b789-4104-9dfa-6105e5b16c8d admin admin] PATH_INFO: `/v3/projects/88aec08b7ce54cb5ad87ea544a819e0a/users/9a882cf9cc15457bb5a860963f1967f2/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:02.899880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b772a0-b789-4104-9dfa-6105e5b16c8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=88aec08b7ce54cb5ad87ea544a819e0a, user_id=9a882cf9cc15457bb5a860963f1967f2, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:02.904720 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b772a0-b789-4104-9dfa-6105e5b16c8d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:02.925322 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1183/2379] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:13:02 2022] PUT /identity/v3/projects/88aec08b7ce54cb5ad87ea544a819e0a/users/9a882cf9cc15457bb5a860963f1967f2/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:13:03.000947 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b66669-0823-422d-a21c-b0e14ae041a7 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:03.001632 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b66669-0823-422d-a21c-b0e14ae041a7 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:03.002201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b66669-0823-422d-a21c-b0e14ae041a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:03.051442 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-64b66669-0823-422d-a21c-b0e14ae041a7 None None] MFA Rules not processed for user `9a882cf9cc15457bb5a860963f1967f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:03.053568 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-64b66669-0823-422d-a21c-b0e14ae041a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:03.168837 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1197/2380] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:13:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:13:03.181994 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b772a0-b789-4104-9dfa-6105e5b16c8d admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:03.184722 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-88b772a0-b789-4104-9dfa-6105e5b16c8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:03.217359 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:03.271695 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b772a0-b789-4104-9dfa-6105e5b16c8d admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:03.275513 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e452739c-c4a8-45a5-9c2f-53927406f6f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:03.276456 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452739c-c4a8-45a5-9c2f-53927406f6f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:03.276740 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452739c-c4a8-45a5-9c2f-53927406f6f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:03.277012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452739c-c4a8-45a5-9c2f-53927406f6f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:03.289623 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e452739c-c4a8-45a5-9c2f-53927406f6f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:03.291089 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e452739c-c4a8-45a5-9c2f-53927406f6f5 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:03.309061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1184/2381] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:13:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:13:04.744244 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edff106-2a14-430a-876b-5928be7c096c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:04.744895 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edff106-2a14-430a-876b-5928be7c096c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:04.744895 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edff106-2a14-430a-876b-5928be7c096c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:04.766449 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-0edff106-2a14-430a-876b-5928be7c096c None None] MFA Rules not processed for user `9a882cf9cc15457bb5a860963f1967f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:04.767691 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0edff106-2a14-430a-876b-5928be7c096c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:04.778572 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1198/2382] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:13:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:13:13.297687 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=17 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:14.764124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:18.366255 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e452739c-c4a8-45a5-9c2f-53927406f6f5 service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:18.368251 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e452739c-c4a8-45a5-9c2f-53927406f6f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:18.495485 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e452739c-c4a8-45a5-9c2f-53927406f6f5 service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:18.502224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308263d9-ca20-4751-9723-4ffe12c10fd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:18.503133 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308263d9-ca20-4751-9723-4ffe12c10fd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:18.503731 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308263d9-ca20-4751-9723-4ffe12c10fd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:18.503731 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308263d9-ca20-4751-9723-4ffe12c10fd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:18.506714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-308263d9-ca20-4751-9723-4ffe12c10fd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:18.673667 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308263d9-ca20-4751-9723-4ffe12c10fd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:18.677695 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308263d9-ca20-4751-9723-4ffe12c10fd1 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:18.698224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1185/2383] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:13:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:13:19.062365 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edff106-2a14-430a-876b-5928be7c096c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:19.077132 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edff106-2a14-430a-876b-5928be7c096c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:19.081169 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2defe51-c6f4-40c0-8d0d-287332e15ab9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:19.082582 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2defe51-c6f4-40c0-8d0d-287332e15ab9 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:19.083082 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2defe51-c6f4-40c0-8d0d-287332e15ab9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:19.083705 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2defe51-c6f4-40c0-8d0d-287332e15ab9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:19.085954 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d2defe51-c6f4-40c0-8d0d-287332e15ab9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:19.108237 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2defe51-c6f4-40c0-8d0d-287332e15ab9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:19.109861 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2defe51-c6f4-40c0-8d0d-287332e15ab9 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:19.122768 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1199/2384] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:13:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:13:22.511584 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22c665-fed0-4c74-ad14-4f5c1b5cb7b8 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:22.512203 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22c665-fed0-4c74-ad14-4f5c1b5cb7b8 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:22.512203 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22c665-fed0-4c74-ad14-4f5c1b5cb7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:22.533569 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-3d22c665-fed0-4c74-ad14-4f5c1b5cb7b8 None None] MFA Rules not processed for user `87f35e960544403ea30ba36d878a86cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:22.535507 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3d22c665-fed0-4c74-ad14-4f5c1b5cb7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:22.603459 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1186/2385] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:13:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:13:23.922547 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa009e23-a7b3-488f-b03b-0dd9dd226f53 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:23.924155 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa009e23-a7b3-488f-b03b-0dd9dd226f53 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:23.924966 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa009e23-a7b3-488f-b03b-0dd9dd226f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:23.954497 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-aa009e23-a7b3-488f-b03b-0dd9dd226f53 None None] MFA Rules not processed for user `e07e83c0bdba47d39c09308eba1bdaae`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:23.956026 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-aa009e23-a7b3-488f-b03b-0dd9dd226f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:24.035671 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1200/2386] 10.222.0.45 () {58 vars in 967 bytes} [Mon Aug 1 14:13:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:13:24.068227 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22c665-fed0-4c74-ad14-4f5c1b5cb7b8 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:24.070310 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3d22c665-fed0-4c74-ad14-4f5c1b5cb7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:24.144574 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:24.176241 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22c665-fed0-4c74-ad14-4f5c1b5cb7b8 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:24.181863 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c70dc8-5c02-41e9-b130-dceecd6c543b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cda7f365ba0405daaf86870960f12d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:24.182610 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c70dc8-5c02-41e9-b130-dceecd6c543b service swift] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:24.183022 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c70dc8-5c02-41e9-b130-dceecd6c543b service swift] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:24.183444 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c70dc8-5c02-41e9-b130-dceecd6c543b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:24.194993 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c70dc8-5c02-41e9-b130-dceecd6c543b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:24.196802 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c70dc8-5c02-41e9-b130-dceecd6c543b service swift] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:24.208087 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1187/2387] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 14:13:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:13:26.195788 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa009e23-a7b3-488f-b03b-0dd9dd226f53 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:26.199637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-aa009e23-a7b3-488f-b03b-0dd9dd226f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:26.218412 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa009e23-a7b3-488f-b03b-0dd9dd226f53 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:26.228830 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b4b5f-0a41-4990-b445-3f96d8193f6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:26.230102 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b4b5f-0a41-4990-b445-3f96d8193f6d service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:26.230569 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b4b5f-0a41-4990-b445-3f96d8193f6d service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:26.231034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b4b5f-0a41-4990-b445-3f96d8193f6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:26.233173 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8d5b4b5f-0a41-4990-b445-3f96d8193f6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:26.250560 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5b4b5f-0a41-4990-b445-3f96d8193f6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:26.252190 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5b4b5f-0a41-4990-b445-3f96d8193f6d service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:26.276568 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1201/2388] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:13:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:13:34.203895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=11 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:34.596226 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e4c48-3322-42f9-8a8a-fa5e73b8e8f2 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:34.596550 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e4c48-3322-42f9-8a8a-fa5e73b8e8f2 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:34.597029 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e4c48-3322-42f9-8a8a-fa5e73b8e8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:34.618759 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-087e4c48-3322-42f9-8a8a-fa5e73b8e8f2 None None] MFA Rules not processed for user `e07e83c0bdba47d39c09308eba1bdaae`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:34.620017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-087e4c48-3322-42f9-8a8a-fa5e73b8e8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:34.628048 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1188/2389] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Aug 1 14:13:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:13:34.640756 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b4b5f-0a41-4990-b445-3f96d8193f6d service nova] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:34.653556 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b4b5f-0a41-4990-b445-3f96d8193f6d service nova] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:34.659673 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab23ad-3cc3-48de-b8e6-bf56c3af89e8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cda7f365ba0405daaf86870960f12d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:34.661491 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab23ad-3cc3-48de-b8e6-bf56c3af89e8 service swift] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:34.661777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab23ad-3cc3-48de-b8e6-bf56c3af89e8 service swift] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:34.662018 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab23ad-3cc3-48de-b8e6-bf56c3af89e8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:34.673061 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab23ad-3cc3-48de-b8e6-bf56c3af89e8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:34.674685 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab23ad-3cc3-48de-b8e6-bf56c3af89e8 service swift] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:34.685906 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1202/2390] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 14:13:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:13:35.321088 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dceeff4-2a29-405d-bcbe-9025d1eed040 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:35.321741 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dceeff4-2a29-405d-bcbe-9025d1eed040 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:35.321741 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dceeff4-2a29-405d-bcbe-9025d1eed040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:35.345277 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-9dceeff4-2a29-405d-bcbe-9025d1eed040 None None] MFA Rules not processed for user `e07e83c0bdba47d39c09308eba1bdaae`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:35.346655 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-9dceeff4-2a29-405d-bcbe-9025d1eed040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:35.354507 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1189/2391] 10.222.0.45 () {58 vars in 967 bytes} [Mon Aug 1 14:13:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:13:35.369719 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab23ad-3cc3-48de-b8e6-bf56c3af89e8 service swift] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:35.381306 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab23ad-3cc3-48de-b8e6-bf56c3af89e8 service swift] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:35.387278 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629bbca8-3077-47d3-8e73-1e8a9345315a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cda7f365ba0405daaf86870960f12d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:35.388084 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bbca8-3077-47d3-8e73-1e8a9345315a service swift] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:35.388352 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bbca8-3077-47d3-8e73-1e8a9345315a service swift] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:35.388627 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bbca8-3077-47d3-8e73-1e8a9345315a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:35.398686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629bbca8-3077-47d3-8e73-1e8a9345315a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:35.400335 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629bbca8-3077-47d3-8e73-1e8a9345315a service swift] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:35.409148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1203/2392] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 14:13:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:13:35.681086 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14a1db-4de0-4ab8-8293-1a0c834a1370 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:35.681373 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14a1db-4de0-4ab8-8293-1a0c834a1370 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:35.681645 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14a1db-4de0-4ab8-8293-1a0c834a1370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:35.704873 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-bf14a1db-4de0-4ab8-8293-1a0c834a1370 None None] MFA Rules not processed for user `e07e83c0bdba47d39c09308eba1bdaae`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:35.706100 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-bf14a1db-4de0-4ab8-8293-1a0c834a1370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:35.713930 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1190/2393] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Aug 1 14:13:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:13:35.726461 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629bbca8-3077-47d3-8e73-1e8a9345315a service swift] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:35.738937 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629bbca8-3077-47d3-8e73-1e8a9345315a service swift] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:35.745313 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5868654d-04ed-4591-8e43-a91048c91dfe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cda7f365ba0405daaf86870960f12d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:35.745853 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868654d-04ed-4591-8e43-a91048c91dfe service swift] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:35.746141 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868654d-04ed-4591-8e43-a91048c91dfe service swift] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:35.746411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868654d-04ed-4591-8e43-a91048c91dfe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:35.758536 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5868654d-04ed-4591-8e43-a91048c91dfe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:35.760148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5868654d-04ed-4591-8e43-a91048c91dfe service swift] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:35.769076 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1204/2394] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 14:13:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:13:36.134286 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fd842-da50-447f-b09e-91ee3066c1dc None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:36.134286 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fd842-da50-447f-b09e-91ee3066c1dc None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:36.134286 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fd842-da50-447f-b09e-91ee3066c1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:36.155759 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-183fd842-da50-447f-b09e-91ee3066c1dc None None] MFA Rules not processed for user `e07e83c0bdba47d39c09308eba1bdaae`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:36.157496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-183fd842-da50-447f-b09e-91ee3066c1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:36.167936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1191/2395] 10.222.0.45 () {58 vars in 967 bytes} [Mon Aug 1 14:13:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:13:36.190122 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5868654d-04ed-4591-8e43-a91048c91dfe service swift] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:36.204222 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5868654d-04ed-4591-8e43-a91048c91dfe service swift] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:36.212711 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc63b2-1d3e-453a-b62b-70272ac803a6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cda7f365ba0405daaf86870960f12d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:36.213739 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc63b2-1d3e-453a-b62b-70272ac803a6 service swift] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:36.214455 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc63b2-1d3e-453a-b62b-70272ac803a6 service swift] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:36.214916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc63b2-1d3e-453a-b62b-70272ac803a6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:36.229699 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efc63b2-1d3e-453a-b62b-70272ac803a6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:36.232640 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efc63b2-1d3e-453a-b62b-70272ac803a6 service swift] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:36.246547 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1205/2396] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 14:13:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:13:38.644099 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fd842-da50-447f-b09e-91ee3066c1dc None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:38.731861 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fd842-da50-447f-b09e-91ee3066c1dc None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:38.737462 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7949f1-594d-4e82-b874-25bf1c77af71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:38.738208 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7949f1-594d-4e82-b874-25bf1c77af71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:38.738525 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7949f1-594d-4e82-b874-25bf1c77af71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:38.738880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7949f1-594d-4e82-b874-25bf1c77af71 admin admin] PATH_INFO: `/v3/users/9a882cf9cc15457bb5a860963f1967f2` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:38.740496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7949f1-594d-4e82-b874-25bf1c77af71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a882cf9cc15457bb5a860963f1967f2)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:38.742721 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7949f1-594d-4e82-b874-25bf1c77af71 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:38.802169 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-be7949f1-594d-4e82-b874-25bf1c77af71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a882cf9cc15457bb5a860963f1967f2'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:13:38.803081 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-be7949f1-594d-4e82-b874-25bf1c77af71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a882cf9cc15457bb5a860963f1967f2'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:13:38.812859 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-be7949f1-594d-4e82-b874-25bf1c77af71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a882cf9cc15457bb5a860963f1967f2'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:13:38.821342 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-be7949f1-594d-4e82-b874-25bf1c77af71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a882cf9cc15457bb5a860963f1967f2'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:13:38.834445 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-be7949f1-594d-4e82-b874-25bf1c77af71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a882cf9cc15457bb5a860963f1967f2'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:13:38.841172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1192/2397] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:13:38 2022] DELETE /identity/v3/users/9a882cf9cc15457bb5a860963f1967f2 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:13:38.846591 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0697dd-f9cf-4185-87cb-66f9fa8eed30 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:38.847346 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0697dd-f9cf-4185-87cb-66f9fa8eed30 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:38.847346 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0697dd-f9cf-4185-87cb-66f9fa8eed30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:38.868698 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-0a0697dd-f9cf-4185-87cb-66f9fa8eed30 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:38.870531 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0a0697dd-f9cf-4185-87cb-66f9fa8eed30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:38.935179 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1206/2398] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:13:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:13:38.948599 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7949f1-594d-4e82-b874-25bf1c77af71 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:38.950169 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-be7949f1-594d-4e82-b874-25bf1c77af71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:39.021955 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7949f1-594d-4e82-b874-25bf1c77af71 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:39.025926 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc396c-88a9-4bfb-97eb-73b278a03701 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:39.026530 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc396c-88a9-4bfb-97eb-73b278a03701 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:39.026823 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc396c-88a9-4bfb-97eb-73b278a03701 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:39.027069 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc396c-88a9-4bfb-97eb-73b278a03701 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:39.035440 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:39.041472 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bc396c-88a9-4bfb-97eb-73b278a03701 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:39.045237 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bc396c-88a9-4bfb-97eb-73b278a03701 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:39.070714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1193/2399] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:13:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:13:39.335878 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0697dd-f9cf-4185-87cb-66f9fa8eed30 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:39.337647 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0a0697dd-f9cf-4185-87cb-66f9fa8eed30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:39.375047 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0697dd-f9cf-4185-87cb-66f9fa8eed30 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:39.379897 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592f221-6506-495e-a743-77ba3c19fec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:39.380602 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592f221-6506-495e-a743-77ba3c19fec5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:39.380885 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592f221-6506-495e-a743-77ba3c19fec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:39.381152 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592f221-6506-495e-a743-77ba3c19fec5 admin admin] PATH_INFO: `/v3/projects/88aec08b7ce54cb5ad87ea544a819e0a` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:39.382418 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1592f221-6506-495e-a743-77ba3c19fec5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88aec08b7ce54cb5ad87ea544a819e0a)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:39.384483 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1592f221-6506-495e-a743-77ba3c19fec5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:39.412759 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:39.453333 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1592f221-6506-495e-a743-77ba3c19fec5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88aec08b7ce54cb5ad87ea544a819e0a', 'request_id': 'req-1592f221-6506-495e-a743-77ba3c19fec5'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:13:39.460961 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1592f221-6506-495e-a743-77ba3c19fec5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88aec08b7ce54cb5ad87ea544a819e0a', 'request_id': 'req-1592f221-6506-495e-a743-77ba3c19fec5'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:13:39.469928 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1592f221-6506-495e-a743-77ba3c19fec5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88aec08b7ce54cb5ad87ea544a819e0a', 'request_id': 'req-1592f221-6506-495e-a743-77ba3c19fec5'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:13:39.484328 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1592f221-6506-495e-a743-77ba3c19fec5 admin admin] The token cache is being invalidate because project 88aec08b7ce54cb5ad87ea544a819e0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:13:39.484625 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1592f221-6506-495e-a743-77ba3c19fec5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:13:39.485405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1592f221-6506-495e-a743-77ba3c19fec5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:13:39.487060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1207/2400] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:13:39 2022] DELETE /identity/v3/projects/88aec08b7ce54cb5ad87ea544a819e0a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:13:40.345660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc396c-88a9-4bfb-97eb-73b278a03701 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:40.347345 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-77bc396c-88a9-4bfb-97eb-73b278a03701 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:40.432822 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc396c-88a9-4bfb-97eb-73b278a03701 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:40.516992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73413333-8a61-4b0d-857e-3a54fa9efd6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:40.517698 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73413333-8a61-4b0d-857e-3a54fa9efd6f service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:40.518537 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73413333-8a61-4b0d-857e-3a54fa9efd6f service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:40.518537 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73413333-8a61-4b0d-857e-3a54fa9efd6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:40.520832 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-73413333-8a61-4b0d-857e-3a54fa9efd6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:40.592083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73413333-8a61-4b0d-857e-3a54fa9efd6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:40.593790 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73413333-8a61-4b0d-857e-3a54fa9efd6f service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:40.604146 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1194/2401] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:13:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:13:42.079647 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869f13b-620e-48b2-94c1-397fef40478f None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:42.080258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869f13b-620e-48b2-94c1-397fef40478f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:42.080258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869f13b-620e-48b2-94c1-397fef40478f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:42.100512 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-3869f13b-620e-48b2-94c1-397fef40478f None None] MFA Rules not processed for user `a3e93dfa85244133b1641a431994b47f`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:42.101746 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3869f13b-620e-48b2-94c1-397fef40478f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:42.109115 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1208/2402] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:13:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:13:47.571846 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73413333-8a61-4b0d-857e-3a54fa9efd6f service nova] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:47.573351 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-73413333-8a61-4b0d-857e-3a54fa9efd6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:47.616142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:47.644885 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73413333-8a61-4b0d-857e-3a54fa9efd6f service nova] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:47.648857 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca88c79-856a-4d44-a516-fad37f3b0800 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:47.650042 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca88c79-856a-4d44-a516-fad37f3b0800 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:47.650360 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca88c79-856a-4d44-a516-fad37f3b0800 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:47.650639 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca88c79-856a-4d44-a516-fad37f3b0800 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:47.652549 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-9ca88c79-856a-4d44-a516-fad37f3b0800 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:47.663058 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca88c79-856a-4d44-a516-fad37f3b0800 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:47.664527 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca88c79-856a-4d44-a516-fad37f3b0800 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:47.672924 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1195/2403] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:13:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:13:52.095387 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:DELETE=7,keystone:SELECT=18,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:57.533629 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869f13b-620e-48b2-94c1-397fef40478f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:57.536798 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3869f13b-620e-48b2-94c1-397fef40478f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:57.581792 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869f13b-620e-48b2-94c1-397fef40478f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:57.590932 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a81be3-1e52-408e-884c-f294a4d3af62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:57.591583 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a81be3-1e52-408e-884c-f294a4d3af62 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:57.591864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a81be3-1e52-408e-884c-f294a4d3af62 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:57.592132 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a81be3-1e52-408e-884c-f294a4d3af62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:57.594419 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b4a81be3-1e52-408e-884c-f294a4d3af62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:57.608578 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a81be3-1e52-408e-884c-f294a4d3af62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:57.610242 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a81be3-1e52-408e-884c-f294a4d3af62 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:57.632526 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1209/2404] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:13:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:13:57.669339 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=12 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:58.933569 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca88c79-856a-4d44-a516-fad37f3b0800 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:58.953283 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca88c79-856a-4d44-a516-fad37f3b0800 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:58.963805 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a247d32-9210-406d-a113-9fb1d1e42f81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:58.965022 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a247d32-9210-406d-a113-9fb1d1e42f81 service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:58.965542 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a247d32-9210-406d-a113-9fb1d1e42f81 service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:58.966039 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a247d32-9210-406d-a113-9fb1d1e42f81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:58.968470 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-2a247d32-9210-406d-a113-9fb1d1e42f81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:58.983434 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a247d32-9210-406d-a113-9fb1d1e42f81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:58.985268 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a247d32-9210-406d-a113-9fb1d1e42f81 service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:58.999309 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1196/2405] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:13:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:14:05.732364 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29dd9f-3479-49b3-b7ac-dedfb3734b7c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:05.732928 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29dd9f-3479-49b3-b7ac-dedfb3734b7c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:05.733335 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29dd9f-3479-49b3-b7ac-dedfb3734b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:05.764369 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-ac29dd9f-3479-49b3-b7ac-dedfb3734b7c None None] MFA Rules not processed for user `e07e83c0bdba47d39c09308eba1bdaae`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:14:05.765996 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ac29dd9f-3479-49b3-b7ac-dedfb3734b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:05.839469 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1210/2406] 10.222.0.45 () {58 vars in 967 bytes} [Mon Aug 1 14:14:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:14:05.854956 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a247d32-9210-406d-a113-9fb1d1e42f81 service nova] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:05.856744 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-2a247d32-9210-406d-a113-9fb1d1e42f81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:05.964358 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a247d32-9210-406d-a113-9fb1d1e42f81 service nova] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:05.971059 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ac9ee-bfc5-4867-bd8c-e7665e241049 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cda7f365ba0405daaf86870960f12d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:05.971717 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ac9ee-bfc5-4867-bd8c-e7665e241049 service swift] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:05.972003 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ac9ee-bfc5-4867-bd8c-e7665e241049 service swift] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:05.972419 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ac9ee-bfc5-4867-bd8c-e7665e241049 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:05.986182 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ac9ee-bfc5-4867-bd8c-e7665e241049 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:05.988839 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ac9ee-bfc5-4867-bd8c-e7665e241049 service swift] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:06.000484 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1197/2407] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 14:14:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:14:15.832160 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=31 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:15.996799 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=38 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:18.538041 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29dd9f-3479-49b3-b7ac-dedfb3734b7c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:18.540484 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ac29dd9f-3479-49b3-b7ac-dedfb3734b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:18.660897 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29dd9f-3479-49b3-b7ac-dedfb3734b7c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:18.668098 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7993631-8a3f-46cb-8ac7-854ce2057610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:18.668821 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7993631-8a3f-46cb-8ac7-854ce2057610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:18.669316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7993631-8a3f-46cb-8ac7-854ce2057610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:18.670370 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7993631-8a3f-46cb-8ac7-854ce2057610 admin admin] PATH_INFO: `/v3/users/ec40d75e772442d2a97e9bf18b4b90c8` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:18.672439 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7993631-8a3f-46cb-8ac7-854ce2057610 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec40d75e772442d2a97e9bf18b4b90c8)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:18.677254 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7993631-8a3f-46cb-8ac7-854ce2057610 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:18.764716 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b7993631-8a3f-46cb-8ac7-854ce2057610 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec40d75e772442d2a97e9bf18b4b90c8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:18.766126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b7993631-8a3f-46cb-8ac7-854ce2057610 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec40d75e772442d2a97e9bf18b4b90c8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:18.782618 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b7993631-8a3f-46cb-8ac7-854ce2057610 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec40d75e772442d2a97e9bf18b4b90c8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:18.792033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b7993631-8a3f-46cb-8ac7-854ce2057610 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec40d75e772442d2a97e9bf18b4b90c8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:18.792845 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b7993631-8a3f-46cb-8ac7-854ce2057610 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec40d75e772442d2a97e9bf18b4b90c8'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:18.827562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1211/2408] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:14:18 2022] DELETE /identity/v3/users/ec40d75e772442d2a97e9bf18b4b90c8 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:14:19.135509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ac9ee-bfc5-4867-bd8c-e7665e241049 service swift] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:19.169454 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ac9ee-bfc5-4867-bd8c-e7665e241049 service swift] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:19.293229 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff99c839-27d4-4f3d-81ec-093c36fca9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:19.294372 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99c839-27d4-4f3d-81ec-093c36fca9cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:19.294862 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99c839-27d4-4f3d-81ec-093c36fca9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:19.295350 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99c839-27d4-4f3d-81ec-093c36fca9cd admin admin] PATH_INFO: `/v3/projects/9d78f1cca2f845a5ad90c514c2d71f62` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:19.297582 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff99c839-27d4-4f3d-81ec-093c36fca9cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d78f1cca2f845a5ad90c514c2d71f62)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:19.301506 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff99c839-27d4-4f3d-81ec-093c36fca9cd admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:19.375492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e036c-2df0-4c29-8d59-bea737e72dd4 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:19.376809 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e036c-2df0-4c29-8d59-bea737e72dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:19.377736 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e036c-2df0-4c29-8d59-bea737e72dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:19.397131 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ff99c839-27d4-4f3d-81ec-093c36fca9cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d78f1cca2f845a5ad90c514c2d71f62', 'request_id': 'req-ff99c839-27d4-4f3d-81ec-093c36fca9cd'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:19.404796 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-1d6e036c-2df0-4c29-8d59-bea737e72dd4 None None] MFA Rules not processed for user `403896b926a1445b9895dc2d8d21854f`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:14:19.406215 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1d6e036c-2df0-4c29-8d59-bea737e72dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:19.406398 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ff99c839-27d4-4f3d-81ec-093c36fca9cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d78f1cca2f845a5ad90c514c2d71f62', 'request_id': 'req-ff99c839-27d4-4f3d-81ec-093c36fca9cd'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:19.417098 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ff99c839-27d4-4f3d-81ec-093c36fca9cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d78f1cca2f845a5ad90c514c2d71f62', 'request_id': 'req-ff99c839-27d4-4f3d-81ec-093c36fca9cd'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:19.435378 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ff99c839-27d4-4f3d-81ec-093c36fca9cd admin admin] The token cache is being invalidate because project 9d78f1cca2f845a5ad90c514c2d71f62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:14:19.435846 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ff99c839-27d4-4f3d-81ec-093c36fca9cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:19.436924 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ff99c839-27d4-4f3d-81ec-093c36fca9cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:19.439110 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1198/2409] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:14:19 2022] DELETE /identity/v3/projects/9d78f1cca2f845a5ad90c514c2d71f62 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:14:19.447826 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff99c839-27d4-4f3d-81ec-093c36fca9cd admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:19.450053 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ff99c839-27d4-4f3d-81ec-093c36fca9cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:19.499291 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1212/2410] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:14:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:14:19.579424 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff99c839-27d4-4f3d-81ec-093c36fca9cd admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:19.587610 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f4038-34c3-4035-8e4a-f75ab93a3f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:19.588819 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f4038-34c3-4035-8e4a-f75ab93a3f5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:19.589337 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f4038-34c3-4035-8e4a-f75ab93a3f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:19.589823 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f4038-34c3-4035-8e4a-f75ab93a3f5c admin admin] PATH_INFO: `/v3/users/ba2b258c27f44ba39e76eff0a8602a95` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:19.592197 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5f4038-34c3-4035-8e4a-f75ab93a3f5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba2b258c27f44ba39e76eff0a8602a95)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:19.596118 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5f4038-34c3-4035-8e4a-f75ab93a3f5c admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:19.656892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:19.683409 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ce5f4038-34c3-4035-8e4a-f75ab93a3f5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba2b258c27f44ba39e76eff0a8602a95'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:19.684527 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ce5f4038-34c3-4035-8e4a-f75ab93a3f5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba2b258c27f44ba39e76eff0a8602a95'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:19.697076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ce5f4038-34c3-4035-8e4a-f75ab93a3f5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba2b258c27f44ba39e76eff0a8602a95'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:19.706435 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ce5f4038-34c3-4035-8e4a-f75ab93a3f5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba2b258c27f44ba39e76eff0a8602a95'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:19.720979 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ce5f4038-34c3-4035-8e4a-f75ab93a3f5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba2b258c27f44ba39e76eff0a8602a95'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:19.728243 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1199/2411] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:14:19 2022] DELETE /identity/v3/users/ba2b258c27f44ba39e76eff0a8602a95 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:14:19.914787 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e036c-2df0-4c29-8d59-bea737e72dd4 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:19.916877 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1d6e036c-2df0-4c29-8d59-bea737e72dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:19.957219 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:20.012875 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e036c-2df0-4c29-8d59-bea737e72dd4 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:20.048775 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f4038-34c3-4035-8e4a-f75ab93a3f5c admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:20.052960 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ce5f4038-34c3-4035-8e4a-f75ab93a3f5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:20.185500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f4038-34c3-4035-8e4a-f75ab93a3f5c admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:20.193720 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb1f15-bc35-4ad0-897b-674f3d7e5049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:20.194687 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb1f15-bc35-4ad0-897b-674f3d7e5049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:20.195087 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb1f15-bc35-4ad0-897b-674f3d7e5049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:20.195470 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb1f15-bc35-4ad0-897b-674f3d7e5049 admin admin] PATH_INFO: `/v3/projects/da9c4987e8034e749a5d73796dfafa5b` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:20.197499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deb1f15-bc35-4ad0-897b-674f3d7e5049 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da9c4987e8034e749a5d73796dfafa5b)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:20.200844 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deb1f15-bc35-4ad0-897b-674f3d7e5049 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:20.280589 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399e2738-2c8b-4be2-825d-b0d6ddc0a3e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:20.281525 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e2738-2c8b-4be2-825d-b0d6ddc0a3e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:20.282043 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e2738-2c8b-4be2-825d-b0d6ddc0a3e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:20.282506 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e2738-2c8b-4be2-825d-b0d6ddc0a3e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:20.284720 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4deb1f15-bc35-4ad0-897b-674f3d7e5049 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da9c4987e8034e749a5d73796dfafa5b', 'request_id': 'req-4deb1f15-bc35-4ad0-897b-674f3d7e5049'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:20.286037 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-399e2738-2c8b-4be2-825d-b0d6ddc0a3e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:20.297324 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4deb1f15-bc35-4ad0-897b-674f3d7e5049 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da9c4987e8034e749a5d73796dfafa5b', 'request_id': 'req-4deb1f15-bc35-4ad0-897b-674f3d7e5049'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:20.303193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399e2738-2c8b-4be2-825d-b0d6ddc0a3e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:20.305845 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399e2738-2c8b-4be2-825d-b0d6ddc0a3e7 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:20.310630 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4deb1f15-bc35-4ad0-897b-674f3d7e5049 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da9c4987e8034e749a5d73796dfafa5b', 'request_id': 'req-4deb1f15-bc35-4ad0-897b-674f3d7e5049'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:20.318895 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1213/2412] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:14:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:14:20.334373 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4deb1f15-bc35-4ad0-897b-674f3d7e5049 admin admin] The token cache is being invalidate because project da9c4987e8034e749a5d73796dfafa5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:14:20.334899 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4deb1f15-bc35-4ad0-897b-674f3d7e5049 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:20.336124 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4deb1f15-bc35-4ad0-897b-674f3d7e5049 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:20.338914 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1200/2413] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:14:20 2022] DELETE /identity/v3/projects/da9c4987e8034e749a5d73796dfafa5b => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:14:20.840729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399e2738-2c8b-4be2-825d-b0d6ddc0a3e7 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:20.842834 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-399e2738-2c8b-4be2-825d-b0d6ddc0a3e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:20.955405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399e2738-2c8b-4be2-825d-b0d6ddc0a3e7 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:21.059942 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8ac7c-06e1-4b74-9d11-e5475fcf3b8a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58380e30789041e4ad9e8652bc628658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:21.060979 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8ac7c-06e1-4b74-9d11-e5475fcf3b8a service placement] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:21.061471 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8ac7c-06e1-4b74-9d11-e5475fcf3b8a service placement] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:21.061962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8ac7c-06e1-4b74-9d11-e5475fcf3b8a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:21.064962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-72d8ac7c-06e1-4b74-9d11-e5475fcf3b8a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:21.085625 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d8ac7c-06e1-4b74-9d11-e5475fcf3b8a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:21.088544 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d8ac7c-06e1-4b74-9d11-e5475fcf3b8a service placement] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:21.105154 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1214/2414] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:14:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:14:30.324809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=68,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:30.534547 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb1f15-bc35-4ad0-897b-674f3d7e5049 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:30.536955 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-4deb1f15-bc35-4ad0-897b-674f3d7e5049 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:30.657140 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb1f15-bc35-4ad0-897b-674f3d7e5049 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:30.664125 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7b525-2aee-49b6-b1eb-44727e9ad9cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:30.664802 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7b525-2aee-49b6-b1eb-44727e9ad9cc service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:30.665080 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7b525-2aee-49b6-b1eb-44727e9ad9cc service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:30.665351 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7b525-2aee-49b6-b1eb-44727e9ad9cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:30.667184 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-aec7b525-2aee-49b6-b1eb-44727e9ad9cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:30.679969 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec7b525-2aee-49b6-b1eb-44727e9ad9cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:30.684009 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec7b525-2aee-49b6-b1eb-44727e9ad9cc service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:30.697520 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1201/2415] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:14:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:14:31.099437 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=93 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:40.688973 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=34 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:52.323343 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8ac7c-06e1-4b74-9d11-e5475fcf3b8a service placement] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:52.324994 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-72d8ac7c-06e1-4b74-9d11-e5475fcf3b8a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:52.384698 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8ac7c-06e1-4b74-9d11-e5475fcf3b8a service placement] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:52.388681 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab7352-229a-4939-b3eb-491a55bb2492 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:52.389344 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab7352-229a-4939-b3eb-491a55bb2492 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:52.389623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab7352-229a-4939-b3eb-491a55bb2492 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:52.389896 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab7352-229a-4939-b3eb-491a55bb2492 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:52.391787 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-86ab7352-229a-4939-b3eb-491a55bb2492 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:52.459642 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab7352-229a-4939-b3eb-491a55bb2492 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:52.461152 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab7352-229a-4939-b3eb-491a55bb2492 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:52.473765 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1215/2416] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:14:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:14:52.601931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b573034-308b-4cc1-b002-9fb8ccc8da2b None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:52.602466 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b573034-308b-4cc1-b002-9fb8ccc8da2b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:52.602559 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b573034-308b-4cc1-b002-9fb8ccc8da2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:52.622810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-4b573034-308b-4cc1-b002-9fb8ccc8da2b None None] MFA Rules not processed for user `8e30910692b4474092f1585a671e2f41`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:14:52.624032 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-4b573034-308b-4cc1-b002-9fb8ccc8da2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:52.684049 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1202/2417] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:14:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:14:53.031156 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab7352-229a-4939-b3eb-491a55bb2492 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:53.032939 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-86ab7352-229a-4939-b3eb-491a55bb2492 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:53.045165 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab7352-229a-4939-b3eb-491a55bb2492 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:53.049251 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a87106a-323e-4cc6-931e-ff1d64587d24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:53.049878 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87106a-323e-4cc6-931e-ff1d64587d24 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:53.050155 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87106a-323e-4cc6-931e-ff1d64587d24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:53.050616 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87106a-323e-4cc6-931e-ff1d64587d24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:53.052374 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3a87106a-323e-4cc6-931e-ff1d64587d24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:53.063192 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a87106a-323e-4cc6-931e-ff1d64587d24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:53.064677 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a87106a-323e-4cc6-931e-ff1d64587d24 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:53.074869 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1216/2418] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:14:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:14:56.521000 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b573034-308b-4cc1-b002-9fb8ccc8da2b None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:56.522510 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-4b573034-308b-4cc1-b002-9fb8ccc8da2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:56.545473 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b573034-308b-4cc1-b002-9fb8ccc8da2b None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:56.553813 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ce5ec-24dd-4c82-a831-27b8d6a6f232 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58380e30789041e4ad9e8652bc628658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:56.555630 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ce5ec-24dd-4c82-a831-27b8d6a6f232 service placement] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:56.556345 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ce5ec-24dd-4c82-a831-27b8d6a6f232 service placement] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:56.556854 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ce5ec-24dd-4c82-a831-27b8d6a6f232 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:56.559565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0f8ce5ec-24dd-4c82-a831-27b8d6a6f232 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:56.581736 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8ce5ec-24dd-4c82-a831-27b8d6a6f232 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:56.583656 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8ce5ec-24dd-4c82-a831-27b8d6a6f232 service placement] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:56.610778 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1203/2419] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:14:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:14:56.629738 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a87106a-323e-4cc6-931e-ff1d64587d24 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:56.631337 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3a87106a-323e-4cc6-931e-ff1d64587d24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:56.650852 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a87106a-323e-4cc6-931e-ff1d64587d24 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:56.682620 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9748d5f-29e7-41ca-9f1e-9e80d16f5593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:56.683870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9748d5f-29e7-41ca-9f1e-9e80d16f5593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:56.684617 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9748d5f-29e7-41ca-9f1e-9e80d16f5593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:56.686671 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9748d5f-29e7-41ca-9f1e-9e80d16f5593 admin admin] PATH_INFO: `/v3/users/a3e93dfa85244133b1641a431994b47f` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:56.688401 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9748d5f-29e7-41ca-9f1e-9e80d16f5593 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3e93dfa85244133b1641a431994b47f)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:56.690607 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9748d5f-29e7-41ca-9f1e-9e80d16f5593 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:56.807529 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a9748d5f-29e7-41ca-9f1e-9e80d16f5593 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3e93dfa85244133b1641a431994b47f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:56.808791 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a9748d5f-29e7-41ca-9f1e-9e80d16f5593 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3e93dfa85244133b1641a431994b47f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:56.828397 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a9748d5f-29e7-41ca-9f1e-9e80d16f5593 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3e93dfa85244133b1641a431994b47f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:56.842641 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a9748d5f-29e7-41ca-9f1e-9e80d16f5593 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3e93dfa85244133b1641a431994b47f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:56.844215 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a9748d5f-29e7-41ca-9f1e-9e80d16f5593 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3e93dfa85244133b1641a431994b47f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:56.882256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1217/2420] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:14:56 2022] DELETE /identity/v3/users/a3e93dfa85244133b1641a431994b47f => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:14:57.620209 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ce5ec-24dd-4c82-a831-27b8d6a6f232 service placement] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:57.632451 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ce5ec-24dd-4c82-a831-27b8d6a6f232 service placement] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:57.725073 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f77b9a-98bd-4fd1-bd45-5afb80f438b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:57.725881 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f77b9a-98bd-4fd1-bd45-5afb80f438b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:57.726166 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f77b9a-98bd-4fd1-bd45-5afb80f438b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:57.726444 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f77b9a-98bd-4fd1-bd45-5afb80f438b6 admin admin] PATH_INFO: `/v3/projects/e85f3e6c18ae4466a1dc3e6464aa18e6` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:57.728465 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f77b9a-98bd-4fd1-bd45-5afb80f438b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e85f3e6c18ae4466a1dc3e6464aa18e6)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:57.730020 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f77b9a-98bd-4fd1-bd45-5afb80f438b6 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:57.791205 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-70f77b9a-98bd-4fd1-bd45-5afb80f438b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e85f3e6c18ae4466a1dc3e6464aa18e6', 'request_id': 'req-70f77b9a-98bd-4fd1-bd45-5afb80f438b6'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:57.803299 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-70f77b9a-98bd-4fd1-bd45-5afb80f438b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e85f3e6c18ae4466a1dc3e6464aa18e6', 'request_id': 'req-70f77b9a-98bd-4fd1-bd45-5afb80f438b6'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:57.814694 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-70f77b9a-98bd-4fd1-bd45-5afb80f438b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e85f3e6c18ae4466a1dc3e6464aa18e6', 'request_id': 'req-70f77b9a-98bd-4fd1-bd45-5afb80f438b6'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:57.830719 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-70f77b9a-98bd-4fd1-bd45-5afb80f438b6 admin admin] The token cache is being invalidate because project e85f3e6c18ae4466a1dc3e6464aa18e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:14:57.830997 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-70f77b9a-98bd-4fd1-bd45-5afb80f438b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:57.831830 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-70f77b9a-98bd-4fd1-bd45-5afb80f438b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:57.833602 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1204/2421] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:14:57 2022] DELETE /identity/v3/projects/e85f3e6c18ae4466a1dc3e6464aa18e6 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:14:57.838979 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b9700-8f0c-452b-bbc3-f2e49b20c901 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:57.839541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b9700-8f0c-452b-bbc3-f2e49b20c901 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:57.839541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b9700-8f0c-452b-bbc3-f2e49b20c901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:57.863645 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-003b9700-8f0c-452b-bbc3-f2e49b20c901 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:14:57.865880 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-003b9700-8f0c-452b-bbc3-f2e49b20c901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:57.872410 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:57.961521 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1218/2422] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:14:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:14:57.974572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e53c6-c944-458b-b6a4-af36502d0ccc None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:57.974962 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e53c6-c944-458b-b6a4-af36502d0ccc None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:57.975305 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e53c6-c944-458b-b6a4-af36502d0ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:57.998413 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-8d8e53c6-c944-458b-b6a4-af36502d0ccc None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:14:57.999768 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-8d8e53c6-c944-458b-b6a4-af36502d0ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:58.009339 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1205/2423] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:14:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:14:58.013891 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b9700-8f0c-452b-bbc3-f2e49b20c901 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:58.027567 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b9700-8f0c-452b-bbc3-f2e49b20c901 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:58.034816 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea3480-e7d2-4a78-8c67-706ba882fe15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:58.035855 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea3480-e7d2-4a78-8c67-706ba882fe15 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:58.036401 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea3480-e7d2-4a78-8c67-706ba882fe15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:58.036934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea3480-e7d2-4a78-8c67-706ba882fe15 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:58.037906 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdea3480-e7d2-4a78-8c67-706ba882fe15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:14:58.038952 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdea3480-e7d2-4a78-8c67-706ba882fe15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:58.041179 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdea3480-e7d2-4a78-8c67-706ba882fe15 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:58.061899 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1219/2424] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:14:58.068499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8e53c6-c944-458b-b6a4-af36502d0ccc None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:58.079438 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8e53c6-c944-458b-b6a4-af36502d0ccc None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:58.085257 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c4236-c04a-4b84-8f2c-baba089432df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:58.085909 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c4236-c04a-4b84-8f2c-baba089432df admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:58.086285 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c4236-c04a-4b84-8f2c-baba089432df admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:58.086676 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c4236-c04a-4b84-8f2c-baba089432df admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:58.087640 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c4236-c04a-4b84-8f2c-baba089432df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:58.089684 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c4236-c04a-4b84-8f2c-baba089432df admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:58.115785 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:58.139888 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1206/2425] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:14:58 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:14:58.146107 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea3480-e7d2-4a78-8c67-706ba882fe15 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:58.168150 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea3480-e7d2-4a78-8c67-706ba882fe15 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:58.295332 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d682a8e-f56a-4b30-b4db-f5fb33eefe9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:58.295978 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d682a8e-f56a-4b30-b4db-f5fb33eefe9d admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:58.296258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d682a8e-f56a-4b30-b4db-f5fb33eefe9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:58.296572 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d682a8e-f56a-4b30-b4db-f5fb33eefe9d admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:58.298978 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d682a8e-f56a-4b30-b4db-f5fb33eefe9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:58.299226 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d682a8e-f56a-4b30-b4db-f5fb33eefe9d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:58.341482 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1220/2426] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:14:58 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:14:58.343746 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c4236-c04a-4b84-8f2c-baba089432df admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:58.379275 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c4236-c04a-4b84-8f2c-baba089432df admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:58.386670 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a79a80-5d42-472f-900c-dca3e4974be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:58.387734 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a79a80-5d42-472f-900c-dca3e4974be9 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:58.389033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a79a80-5d42-472f-900c-dca3e4974be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:58.390201 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a79a80-5d42-472f-900c-dca3e4974be9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:58.391432 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a79a80-5d42-472f-900c-dca3e4974be9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:58.393371 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a79a80-5d42-472f-900c-dca3e4974be9 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:58.426119 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1207/2427] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:14:58.430624 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d682a8e-f56a-4b30-b4db-f5fb33eefe9d admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:58.448653 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d682a8e-f56a-4b30-b4db-f5fb33eefe9d admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:58.459728 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b65385-dc4c-42b3-987a-32bd206e453f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:58.460996 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b65385-dc4c-42b3-987a-32bd206e453f admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:58.461529 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b65385-dc4c-42b3-987a-32bd206e453f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:58.462009 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b65385-dc4c-42b3-987a-32bd206e453f admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:58.463988 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b65385-dc4c-42b3-987a-32bd206e453f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:58.467849 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b65385-dc4c-42b3-987a-32bd206e453f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:58.493512 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1221/2428] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:14:58.494891 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a79a80-5d42-472f-900c-dca3e4974be9 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:58.524836 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a79a80-5d42-472f-900c-dca3e4974be9 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:58.546999 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb5569-828e-4adc-aa26-b9ab9f1040d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:58.548035 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb5569-828e-4adc-aa26-b9ab9f1040d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:58.548442 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb5569-828e-4adc-aa26-b9ab9f1040d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:58.548813 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb5569-828e-4adc-aa26-b9ab9f1040d8 admin admin] PATH_INFO: `/v3/projects/db87d8b3358945aa88acbdc4246917ae/users/a9fc33bdca304c33b6a78122a54f492a/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:58.591228 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb5569-828e-4adc-aa26-b9ab9f1040d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db87d8b3358945aa88acbdc4246917ae, user_id=a9fc33bdca304c33b6a78122a54f492a, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:58.598303 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb5569-828e-4adc-aa26-b9ab9f1040d8 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:58.646981 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1208/2429] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:14:58 2022] PUT /identity/v3/projects/db87d8b3358945aa88acbdc4246917ae/users/a9fc33bdca304c33b6a78122a54f492a/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:14:58.647971 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf7498-32e4-4d8c-bfdf-28a576d01586 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:58.648367 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf7498-32e4-4d8c-bfdf-28a576d01586 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:58.648631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf7498-32e4-4d8c-bfdf-28a576d01586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:58.692133 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-afdf7498-32e4-4d8c-bfdf-28a576d01586 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:14:58.693660 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-afdf7498-32e4-4d8c-bfdf-28a576d01586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:58.791871 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1222/2430] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:14:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:14:58.802533 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb5569-828e-4adc-aa26-b9ab9f1040d8 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:58.804943 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-6abb5569-828e-4adc-aa26-b9ab9f1040d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:58.872769 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb5569-828e-4adc-aa26-b9ab9f1040d8 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:58.878154 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ededf09-4b03-4f3a-a384-6a3653c1b510 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:58.879300 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ededf09-4b03-4f3a-a384-6a3653c1b510 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:58.879595 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ededf09-4b03-4f3a-a384-6a3653c1b510 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:58.879871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ededf09-4b03-4f3a-a384-6a3653c1b510 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:58.892251 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ededf09-4b03-4f3a-a384-6a3653c1b510 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:58.893769 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ededf09-4b03-4f3a-a384-6a3653c1b510 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:58.906036 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1209/2431] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:14:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:15:05.515673 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3bf07b-38b5-4b4c-bcc0-b0be886c7eaf None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:05.516309 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3bf07b-38b5-4b4c-bcc0-b0be886c7eaf None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:05.516309 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3bf07b-38b5-4b4c-bcc0-b0be886c7eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:05.552679 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-db3bf07b-38b5-4b4c-bcc0-b0be886c7eaf None None] MFA Rules not processed for user `a9fc33bdca304c33b6a78122a54f492a`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:05.554716 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-db3bf07b-38b5-4b4c-bcc0-b0be886c7eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:05.567836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:05.657118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1223/2432] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:15:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:15:05.690218 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ededf09-4b03-4f3a-a384-6a3653c1b510 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:05.691906 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3ededf09-4b03-4f3a-a384-6a3653c1b510 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:05.819541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ededf09-4b03-4f3a-a384-6a3653c1b510 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:05.845712 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc51aeac-7cf6-4ac5-b6c1-270d26db7973 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:05.846772 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51aeac-7cf6-4ac5-b6c1-270d26db7973 service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:05.847330 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51aeac-7cf6-4ac5-b6c1-270d26db7973 service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:05.847958 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51aeac-7cf6-4ac5-b6c1-270d26db7973 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:05.863110 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc51aeac-7cf6-4ac5-b6c1-270d26db7973 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:05.865783 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc51aeac-7cf6-4ac5-b6c1-270d26db7973 service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:05.884307 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1210/2433] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:15:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:15:05.943177 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3bf07b-38b5-4b4c-bcc0-b0be886c7eaf None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:05.957157 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3bf07b-38b5-4b4c-bcc0-b0be886c7eaf None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:05.980147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046fe212-dbc1-4661-bc9e-eaf2717336b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:05.981247 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fe212-dbc1-4661-bc9e-eaf2717336b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:05.981737 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fe212-dbc1-4661-bc9e-eaf2717336b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:05.982239 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fe212-dbc1-4661-bc9e-eaf2717336b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:05.986219 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-046fe212-dbc1-4661-bc9e-eaf2717336b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:06.017413 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046fe212-dbc1-4661-bc9e-eaf2717336b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:06.021449 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046fe212-dbc1-4661-bc9e-eaf2717336b3 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:06.041965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1224/2434] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:15:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:15:07.905475 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6f4dc-191f-4981-a828-a505bf554373 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:07.906049 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6f4dc-191f-4981-a828-a505bf554373 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:07.906403 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6f4dc-191f-4981-a828-a505bf554373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:07.944244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-94e6f4dc-191f-4981-a828-a505bf554373 None None] MFA Rules not processed for user `a9fc33bdca304c33b6a78122a54f492a`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:07.946365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-94e6f4dc-191f-4981-a828-a505bf554373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:07.972246 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1211/2435] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:15:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:15:12.216874 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046fe212-dbc1-4661-bc9e-eaf2717336b3 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:12.221777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-046fe212-dbc1-4661-bc9e-eaf2717336b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:12.252340 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046fe212-dbc1-4661-bc9e-eaf2717336b3 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:12.259816 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c538a8c-8ef1-4c51-bad5-ee556c72eb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:12.261760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c538a8c-8ef1-4c51-bad5-ee556c72eb56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:12.262265 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c538a8c-8ef1-4c51-bad5-ee556c72eb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:12.262786 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c538a8c-8ef1-4c51-bad5-ee556c72eb56 admin admin] PATH_INFO: `/v3/users/87f35e960544403ea30ba36d878a86cb` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:12.264743 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c538a8c-8ef1-4c51-bad5-ee556c72eb56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87f35e960544403ea30ba36d878a86cb)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:12.268947 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c538a8c-8ef1-4c51-bad5-ee556c72eb56 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:12.364689 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-0c538a8c-8ef1-4c51-bad5-ee556c72eb56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87f35e960544403ea30ba36d878a86cb'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:12.369826 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-0c538a8c-8ef1-4c51-bad5-ee556c72eb56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87f35e960544403ea30ba36d878a86cb'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:12.396864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-0c538a8c-8ef1-4c51-bad5-ee556c72eb56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87f35e960544403ea30ba36d878a86cb'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:12.418940 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-0c538a8c-8ef1-4c51-bad5-ee556c72eb56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87f35e960544403ea30ba36d878a86cb'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:12.420155 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-0c538a8c-8ef1-4c51-bad5-ee556c72eb56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87f35e960544403ea30ba36d878a86cb'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:12.449599 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1225/2436] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:15:12 2022] DELETE /identity/v3/users/87f35e960544403ea30ba36d878a86cb => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:15:12.675557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e6f4dc-191f-4981-a828-a505bf554373 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:12.724911 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e6f4dc-191f-4981-a828-a505bf554373 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:12.802686 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:12.835469 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad84d681-b4b5-4093-9f50-b04ed040e2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:12.836264 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad84d681-b4b5-4093-9f50-b04ed040e2f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:12.836648 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad84d681-b4b5-4093-9f50-b04ed040e2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:12.837032 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad84d681-b4b5-4093-9f50-b04ed040e2f3 admin admin] PATH_INFO: `/v3/projects/1081ae5fd3e9443c80626517ae9c7d77` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:12.838758 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad84d681-b4b5-4093-9f50-b04ed040e2f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1081ae5fd3e9443c80626517ae9c7d77)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:12.841179 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad84d681-b4b5-4093-9f50-b04ed040e2f3 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:12.910073 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ad84d681-b4b5-4093-9f50-b04ed040e2f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1081ae5fd3e9443c80626517ae9c7d77', 'request_id': 'req-ad84d681-b4b5-4093-9f50-b04ed040e2f3'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:12.919775 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ad84d681-b4b5-4093-9f50-b04ed040e2f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1081ae5fd3e9443c80626517ae9c7d77', 'request_id': 'req-ad84d681-b4b5-4093-9f50-b04ed040e2f3'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:12.935784 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ad84d681-b4b5-4093-9f50-b04ed040e2f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1081ae5fd3e9443c80626517ae9c7d77', 'request_id': 'req-ad84d681-b4b5-4093-9f50-b04ed040e2f3'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:12.967832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c538a8c-8ef1-4c51-bad5-ee556c72eb56 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:12.969358 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0c538a8c-8ef1-4c51-bad5-ee556c72eb56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:12.978966 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ad84d681-b4b5-4093-9f50-b04ed040e2f3 admin admin] The token cache is being invalidate because project 1081ae5fd3e9443c80626517ae9c7d77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:15:12.979625 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ad84d681-b4b5-4093-9f50-b04ed040e2f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:12.982635 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ad84d681-b4b5-4093-9f50-b04ed040e2f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:12.998230 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1212/2437] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:15:12 2022] DELETE /identity/v3/projects/1081ae5fd3e9443c80626517ae9c7d77 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:15:13.002334 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28516624-57b7-450d-86bd-e8f519a82427 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:13.003136 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28516624-57b7-450d-86bd-e8f519a82427 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:13.003971 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28516624-57b7-450d-86bd-e8f519a82427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:13.033653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-28516624-57b7-450d-86bd-e8f519a82427 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:13.037512 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-28516624-57b7-450d-86bd-e8f519a82427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:13.185633 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c538a8c-8ef1-4c51-bad5-ee556c72eb56 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:13.187401 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1213/2438] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:15:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:15:13.201321 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4e094-4045-41c7-8418-2b2197894f1f None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:13.202434 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4e094-4045-41c7-8418-2b2197894f1f None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:13.203411 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4e094-4045-41c7-8418-2b2197894f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:13.238878 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:13.264163 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-9ac4e094-4045-41c7-8418-2b2197894f1f None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:13.265630 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-9ac4e094-4045-41c7-8418-2b2197894f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:13.275864 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1214/2439] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:15:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:15:13.281982 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4e094-4045-41c7-8418-2b2197894f1f None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:13.309709 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4e094-4045-41c7-8418-2b2197894f1f None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:13.316000 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6209e8c-93fe-4541-9795-16d01b4b4b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:13.316844 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6209e8c-93fe-4541-9795-16d01b4b4b1f admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:13.317223 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6209e8c-93fe-4541-9795-16d01b4b4b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:13.317905 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6209e8c-93fe-4541-9795-16d01b4b4b1f admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:13.319836 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6209e8c-93fe-4541-9795-16d01b4b4b1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:15:13.320315 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6209e8c-93fe-4541-9795-16d01b4b4b1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:13.322090 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445855ac-d050-4cc6-9ea9-572f293aeac0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:13.322729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6209e8c-93fe-4541-9795-16d01b4b4b1f admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:13.324188 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445855ac-d050-4cc6-9ea9-572f293aeac0 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:13.324995 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445855ac-d050-4cc6-9ea9-572f293aeac0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:13.326030 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445855ac-d050-4cc6-9ea9-572f293aeac0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:13.333142 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-445855ac-d050-4cc6-9ea9-572f293aeac0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:13.358227 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1215/2440] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:15:13.366028 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6209e8c-93fe-4541-9795-16d01b4b4b1f admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:13.378713 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6209e8c-93fe-4541-9795-16d01b4b4b1f admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:13.384892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6310a4-edb5-4eac-a37f-f4d4de23572a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:13.385555 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6310a4-edb5-4eac-a37f-f4d4de23572a admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:13.385857 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6310a4-edb5-4eac-a37f-f4d4de23572a admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:13.386120 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6310a4-edb5-4eac-a37f-f4d4de23572a admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:13.386797 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6310a4-edb5-4eac-a37f-f4d4de23572a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:13.389061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6310a4-edb5-4eac-a37f-f4d4de23572a admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:13.455430 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445855ac-d050-4cc6-9ea9-572f293aeac0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:13.456991 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1216/2441] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:15:13 2022] POST /identity/v3/projects => generated 381 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:15:13.461699 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445855ac-d050-4cc6-9ea9-572f293aeac0 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:13.462385 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6310a4-edb5-4eac-a37f-f4d4de23572a admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:13.492016 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1226/2442] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:15:12 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:15:13.591706 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:13.662814 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6310a4-edb5-4eac-a37f-f4d4de23572a admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:13.675191 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf461fa-cef6-4eb4-870a-e68f48c2855c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:13.676318 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf461fa-cef6-4eb4-870a-e68f48c2855c admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:13.677369 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf461fa-cef6-4eb4-870a-e68f48c2855c admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:13.677826 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf461fa-cef6-4eb4-870a-e68f48c2855c admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:13.679291 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf461fa-cef6-4eb4-870a-e68f48c2855c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:13.698278 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf461fa-cef6-4eb4-870a-e68f48c2855c admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:13.731557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1217/2443] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:15:13 2022] POST /identity/v3/users => generated 291 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:15:13.736826 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445855ac-d050-4cc6-9ea9-572f293aeac0 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:13.769162 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445855ac-d050-4cc6-9ea9-572f293aeac0 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:13.784427 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b903f-072e-4306-a57d-89f5cb2798da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:13.785604 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b903f-072e-4306-a57d-89f5cb2798da admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:13.786016 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b903f-072e-4306-a57d-89f5cb2798da admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:13.800728 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b903f-072e-4306-a57d-89f5cb2798da admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:13.800728 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50b903f-072e-4306-a57d-89f5cb2798da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:13.800728 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50b903f-072e-4306-a57d-89f5cb2798da admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:13.821254 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1227/2444] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:15:13.829684 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf461fa-cef6-4eb4-870a-e68f48c2855c admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:13.862221 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf461fa-cef6-4eb4-870a-e68f48c2855c admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:13.878132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4496a-66df-4ce8-83bb-e4952952a504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:13.879352 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4496a-66df-4ce8-83bb-e4952952a504 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:13.879669 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4496a-66df-4ce8-83bb-e4952952a504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:13.879943 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4496a-66df-4ce8-83bb-e4952952a504 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:13.886035 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4496a-66df-4ce8-83bb-e4952952a504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:13.890723 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4496a-66df-4ce8-83bb-e4952952a504 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:13.917267 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1218/2445] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:15:13.918143 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b903f-072e-4306-a57d-89f5cb2798da admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:13.931962 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b903f-072e-4306-a57d-89f5cb2798da admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:13.958400 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc79c0-7ee0-4f5c-a49e-94bf82df768c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:13.959153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc79c0-7ee0-4f5c-a49e-94bf82df768c admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:13.959432 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc79c0-7ee0-4f5c-a49e-94bf82df768c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:13.959705 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc79c0-7ee0-4f5c-a49e-94bf82df768c admin admin] PATH_INFO: `/v3/projects/4ebac7e944724e3da40032c45b57d095/users/8798612a4dba43aca545f3f2faa39922/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:14.002088 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc79c0-7ee0-4f5c-a49e-94bf82df768c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ebac7e944724e3da40032c45b57d095, user_id=8798612a4dba43aca545f3f2faa39922, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:14.012838 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc79c0-7ee0-4f5c-a49e-94bf82df768c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:14.040321 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1228/2446] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:15:13 2022] PUT /identity/v3/projects/4ebac7e944724e3da40032c45b57d095/users/8798612a4dba43aca545f3f2faa39922/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:15:14.056476 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e26c1-56f6-468c-bc9b-ad743e089d91 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:14.056941 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e26c1-56f6-468c-bc9b-ad743e089d91 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:14.057382 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e26c1-56f6-468c-bc9b-ad743e089d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:14.117345 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-0d1e26c1-56f6-468c-bc9b-ad743e089d91 None None] MFA Rules not processed for user `8798612a4dba43aca545f3f2faa39922`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:14.120045 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0d1e26c1-56f6-468c-bc9b-ad743e089d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:14.288591 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1219/2447] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:15:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:15:14.292723 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc79c0-7ee0-4f5c-a49e-94bf82df768c admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:14.393345 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc79c0-7ee0-4f5c-a49e-94bf82df768c admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:14.400002 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99cd5c1-a8ca-4b93-a11c-8b24c284f322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:14.401063 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99cd5c1-a8ca-4b93-a11c-8b24c284f322 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:14.401911 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99cd5c1-a8ca-4b93-a11c-8b24c284f322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:14.402209 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99cd5c1-a8ca-4b93-a11c-8b24c284f322 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:14.402916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99cd5c1-a8ca-4b93-a11c-8b24c284f322 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:14.404958 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99cd5c1-a8ca-4b93-a11c-8b24c284f322 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:14.453036 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1229/2448] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:15:14 2022] POST /identity/v3/projects => generated 381 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:15:14.454789 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e26c1-56f6-468c-bc9b-ad743e089d91 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:14.471751 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e26c1-56f6-468c-bc9b-ad743e089d91 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:14.609518 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8fec2f-ef77-4c24-813b-03372c92b215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:14.610184 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fec2f-ef77-4c24-813b-03372c92b215 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:14.610469 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fec2f-ef77-4c24-813b-03372c92b215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:14.610742 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fec2f-ef77-4c24-813b-03372c92b215 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:14.611441 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8fec2f-ef77-4c24-813b-03372c92b215 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:14.613532 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8fec2f-ef77-4c24-813b-03372c92b215 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:14.648310 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1220/2449] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:15:14 2022] POST /identity/v3/users => generated 297 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:15:14.652646 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99cd5c1-a8ca-4b93-a11c-8b24c284f322 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:14.680508 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99cd5c1-a8ca-4b93-a11c-8b24c284f322 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:14.689535 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b63836-82c1-4624-8eb5-67395ea1f237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:14.690796 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b63836-82c1-4624-8eb5-67395ea1f237 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:14.691280 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b63836-82c1-4624-8eb5-67395ea1f237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:14.691714 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b63836-82c1-4624-8eb5-67395ea1f237 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:14.694681 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b63836-82c1-4624-8eb5-67395ea1f237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:14.698420 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b63836-82c1-4624-8eb5-67395ea1f237 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:14.720123 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1230/2450] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:15:14.725366 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8fec2f-ef77-4c24-813b-03372c92b215 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:14.738154 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8fec2f-ef77-4c24-813b-03372c92b215 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:14.744895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cab2d6-1529-48b0-9f3f-14ee8d450c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:14.745505 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cab2d6-1529-48b0-9f3f-14ee8d450c9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:14.745953 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cab2d6-1529-48b0-9f3f-14ee8d450c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:14.746233 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cab2d6-1529-48b0-9f3f-14ee8d450c9f admin admin] PATH_INFO: `/v3/projects/24fe3482c42347a3b533219a4d8d30b7/users/29df221d65504ab29b651aa5541c5eb5/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:14.770707 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cab2d6-1529-48b0-9f3f-14ee8d450c9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=24fe3482c42347a3b533219a4d8d30b7, user_id=29df221d65504ab29b651aa5541c5eb5, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:14.774904 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cab2d6-1529-48b0-9f3f-14ee8d450c9f admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:14.804845 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1221/2451] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:15:14 2022] PUT /identity/v3/projects/24fe3482c42347a3b533219a4d8d30b7/users/29df221d65504ab29b651aa5541c5eb5/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:15:14.809923 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b63836-82c1-4624-8eb5-67395ea1f237 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:14.833065 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b63836-82c1-4624-8eb5-67395ea1f237 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:14.867591 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:15.007776 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6cbb91-81b8-43db-b7f7-b6e7f88fc3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:15.008923 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6cbb91-81b8-43db-b7f7-b6e7f88fc3b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:15.009385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6cbb91-81b8-43db-b7f7-b6e7f88fc3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:15.010021 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6cbb91-81b8-43db-b7f7-b6e7f88fc3b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:15.011339 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6cbb91-81b8-43db-b7f7-b6e7f88fc3b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:15.014797 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6cbb91-81b8-43db-b7f7-b6e7f88fc3b0 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:15.049014 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1231/2452] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:15:15.068350 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cab2d6-1529-48b0-9f3f-14ee8d450c9f admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:15.068350 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:15.088966 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cab2d6-1529-48b0-9f3f-14ee8d450c9f admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:15.098525 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbea6fd-aeb9-4851-adf7-a15b301627c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:15.099933 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbea6fd-aeb9-4851-adf7-a15b301627c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:15.100645 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbea6fd-aeb9-4851-adf7-a15b301627c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:15.101144 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbea6fd-aeb9-4851-adf7-a15b301627c8 admin admin] PATH_INFO: `/v3/projects/24fe3482c42347a3b533219a4d8d30b7/users/29df221d65504ab29b651aa5541c5eb5/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:15.105202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbea6fd-aeb9-4851-adf7-a15b301627c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24fe3482c42347a3b533219a4d8d30b7, user_id=29df221d65504ab29b651aa5541c5eb5, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:15.111605 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbea6fd-aeb9-4851-adf7-a15b301627c8 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:15.158938 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1222/2453] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:15:15 2022] PUT /identity/v3/projects/24fe3482c42347a3b533219a4d8d30b7/users/29df221d65504ab29b651aa5541c5eb5/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:15:15.178940 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891347da-a3a2-468e-bed0-2c7140470bc2 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:15.178940 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891347da-a3a2-468e-bed0-2c7140470bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:15.179475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891347da-a3a2-468e-bed0-2c7140470bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:15.234736 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-891347da-a3a2-468e-bed0-2c7140470bc2 None None] MFA Rules not processed for user `29df221d65504ab29b651aa5541c5eb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:15.236736 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-891347da-a3a2-468e-bed0-2c7140470bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:15.369162 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1232/2454] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:15:15 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:15:15.385135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbea6fd-aeb9-4851-adf7-a15b301627c8 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:15.387968 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-6bbea6fd-aeb9-4851-adf7-a15b301627c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:15.565892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbea6fd-aeb9-4851-adf7-a15b301627c8 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:15.571404 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c47c5-2747-499f-be63-240df40ea7fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:15.571993 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c47c5-2747-499f-be63-240df40ea7fd service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:15.572294 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c47c5-2747-499f-be63-240df40ea7fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:15.572858 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c47c5-2747-499f-be63-240df40ea7fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:15.586824 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c47c5-2747-499f-be63-240df40ea7fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:15.588434 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c47c5-2747-499f-be63-240df40ea7fd service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:15.612734 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1223/2455] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:15:15 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:15:15.728626 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891347da-a3a2-468e-bed0-2c7140470bc2 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:15.741359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891347da-a3a2-468e-bed0-2c7140470bc2 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:15.746624 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f7c545-4cfe-4975-bdc1-fe859ce2782a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:15.747731 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f7c545-4cfe-4975-bdc1-fe859ce2782a service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:15.748431 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f7c545-4cfe-4975-bdc1-fe859ce2782a service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:15.748873 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f7c545-4cfe-4975-bdc1-fe859ce2782a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:15.836453 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f7c545-4cfe-4975-bdc1-fe859ce2782a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:15.838945 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f7c545-4cfe-4975-bdc1-fe859ce2782a service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:15.855793 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1233/2456] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:15:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:15:25.314323 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c47c5-2747-499f-be63-240df40ea7fd service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:25.395990 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c47c5-2747-499f-be63-240df40ea7fd service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:25.401823 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be752cd4-cb75-4338-8e42-947f1ab89b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:25.402618 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be752cd4-cb75-4338-8e42-947f1ab89b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:25.403198 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be752cd4-cb75-4338-8e42-947f1ab89b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:25.403506 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be752cd4-cb75-4338-8e42-947f1ab89b73 admin admin] PATH_INFO: `/v3/users/8798612a4dba43aca545f3f2faa39922` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:25.405238 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be752cd4-cb75-4338-8e42-947f1ab89b73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8798612a4dba43aca545f3f2faa39922)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:25.407737 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be752cd4-cb75-4338-8e42-947f1ab89b73 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:25.493638 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-be752cd4-cb75-4338-8e42-947f1ab89b73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8798612a4dba43aca545f3f2faa39922'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:25.494880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-be752cd4-cb75-4338-8e42-947f1ab89b73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8798612a4dba43aca545f3f2faa39922'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:25.507364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-be752cd4-cb75-4338-8e42-947f1ab89b73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8798612a4dba43aca545f3f2faa39922'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:25.517431 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-be752cd4-cb75-4338-8e42-947f1ab89b73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8798612a4dba43aca545f3f2faa39922'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:25.531735 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-be752cd4-cb75-4338-8e42-947f1ab89b73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8798612a4dba43aca545f3f2faa39922'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:25.541830 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1224/2457] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:15:25 2022] DELETE /identity/v3/users/8798612a4dba43aca545f3f2faa39922 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:15:25.547787 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92796cf5-b034-42ef-ac43-2ea51f126f29 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:25.548419 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92796cf5-b034-42ef-ac43-2ea51f126f29 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:25.548419 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92796cf5-b034-42ef-ac43-2ea51f126f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:25.571335 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-92796cf5-b034-42ef-ac43-2ea51f126f29 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:25.572624 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-92796cf5-b034-42ef-ac43-2ea51f126f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:25.616899 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:25.658492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1234/2458] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:15:25.670855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be752cd4-cb75-4338-8e42-947f1ab89b73 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:25.673029 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-be752cd4-cb75-4338-8e42-947f1ab89b73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:25.690905 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:25.786484 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be752cd4-cb75-4338-8e42-947f1ab89b73 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:25.803225 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0683741c-bd65-4d4d-ab06-5217576eaf09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:25.804456 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683741c-bd65-4d4d-ab06-5217576eaf09 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:25.805470 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683741c-bd65-4d4d-ab06-5217576eaf09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:25.805998 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683741c-bd65-4d4d-ab06-5217576eaf09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:25.820395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0683741c-bd65-4d4d-ab06-5217576eaf09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:25.821973 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0683741c-bd65-4d4d-ab06-5217576eaf09 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:25.836807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1225/2459] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:15:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:15:26.299787 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92796cf5-b034-42ef-ac43-2ea51f126f29 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:26.302500 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-92796cf5-b034-42ef-ac43-2ea51f126f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:26.318231 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92796cf5-b034-42ef-ac43-2ea51f126f29 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:26.325450 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a78d6-b626-4f9e-9c49-621a97f2c9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:26.326393 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a78d6-b626-4f9e-9c49-621a97f2c9c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:26.327597 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a78d6-b626-4f9e-9c49-621a97f2c9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:26.327971 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a78d6-b626-4f9e-9c49-621a97f2c9c1 admin admin] PATH_INFO: `/v3/projects/4ebac7e944724e3da40032c45b57d095` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:26.329593 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a78d6-b626-4f9e-9c49-621a97f2c9c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ebac7e944724e3da40032c45b57d095)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:26.331849 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a78d6-b626-4f9e-9c49-621a97f2c9c1 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:26.393737 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fe4a78d6-b626-4f9e-9c49-621a97f2c9c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ebac7e944724e3da40032c45b57d095', 'request_id': 'req-fe4a78d6-b626-4f9e-9c49-621a97f2c9c1'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:26.403224 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fe4a78d6-b626-4f9e-9c49-621a97f2c9c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ebac7e944724e3da40032c45b57d095', 'request_id': 'req-fe4a78d6-b626-4f9e-9c49-621a97f2c9c1'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:26.416635 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fe4a78d6-b626-4f9e-9c49-621a97f2c9c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ebac7e944724e3da40032c45b57d095', 'request_id': 'req-fe4a78d6-b626-4f9e-9c49-621a97f2c9c1'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:26.431844 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fe4a78d6-b626-4f9e-9c49-621a97f2c9c1 admin admin] The token cache is being invalidate because project 4ebac7e944724e3da40032c45b57d095 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:15:26.432175 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fe4a78d6-b626-4f9e-9c49-621a97f2c9c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:26.432833 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fe4a78d6-b626-4f9e-9c49-621a97f2c9c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:26.434727 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1235/2460] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:15:26 2022] DELETE /identity/v3/projects/4ebac7e944724e3da40032c45b57d095 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:15:26.443787 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0683741c-bd65-4d4d-ab06-5217576eaf09 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:26.461328 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0683741c-bd65-4d4d-ab06-5217576eaf09 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:26.551937 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca28310e-4a20-45ce-ae89-31e5c0f45899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:26.553383 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28310e-4a20-45ce-ae89-31e5c0f45899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:26.553965 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28310e-4a20-45ce-ae89-31e5c0f45899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:26.554430 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28310e-4a20-45ce-ae89-31e5c0f45899 admin admin] PATH_INFO: `/v3/users/29df221d65504ab29b651aa5541c5eb5` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:26.557002 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca28310e-4a20-45ce-ae89-31e5c0f45899 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29df221d65504ab29b651aa5541c5eb5)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:26.559554 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca28310e-4a20-45ce-ae89-31e5c0f45899 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:26.632822 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ca28310e-4a20-45ce-ae89-31e5c0f45899 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29df221d65504ab29b651aa5541c5eb5'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:26.633618 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ca28310e-4a20-45ce-ae89-31e5c0f45899 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29df221d65504ab29b651aa5541c5eb5'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:26.646246 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ca28310e-4a20-45ce-ae89-31e5c0f45899 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29df221d65504ab29b651aa5541c5eb5'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:26.655350 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ca28310e-4a20-45ce-ae89-31e5c0f45899 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29df221d65504ab29b651aa5541c5eb5'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:26.670481 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ca28310e-4a20-45ce-ae89-31e5c0f45899 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29df221d65504ab29b651aa5541c5eb5'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:26.678213 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1226/2461] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:15:26 2022] DELETE /identity/v3/users/29df221d65504ab29b651aa5541c5eb5 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:15:27.163236 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a78d6-b626-4f9e-9c49-621a97f2c9c1 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:27.165707 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fe4a78d6-b626-4f9e-9c49-621a97f2c9c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:27.186134 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a78d6-b626-4f9e-9c49-621a97f2c9c1 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:27.279205 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e03a2-2364-4d89-960d-d0a2dbb4ef3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:27.279823 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e03a2-2364-4d89-960d-d0a2dbb4ef3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:27.280326 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e03a2-2364-4d89-960d-d0a2dbb4ef3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:27.281110 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e03a2-2364-4d89-960d-d0a2dbb4ef3f admin admin] PATH_INFO: `/v3/projects/24fe3482c42347a3b533219a4d8d30b7` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:27.282223 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e03a2-2364-4d89-960d-d0a2dbb4ef3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=24fe3482c42347a3b533219a4d8d30b7)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:27.284856 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e03a2-2364-4d89-960d-d0a2dbb4ef3f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:27.374420 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a45e03a2-2364-4d89-960d-d0a2dbb4ef3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24fe3482c42347a3b533219a4d8d30b7', 'request_id': 'req-a45e03a2-2364-4d89-960d-d0a2dbb4ef3f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:27.386887 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a45e03a2-2364-4d89-960d-d0a2dbb4ef3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24fe3482c42347a3b533219a4d8d30b7', 'request_id': 'req-a45e03a2-2364-4d89-960d-d0a2dbb4ef3f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:27.390798 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:27.402834 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a45e03a2-2364-4d89-960d-d0a2dbb4ef3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24fe3482c42347a3b533219a4d8d30b7', 'request_id': 'req-a45e03a2-2364-4d89-960d-d0a2dbb4ef3f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:27.435540 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a45e03a2-2364-4d89-960d-d0a2dbb4ef3f admin admin] The token cache is being invalidate because project 24fe3482c42347a3b533219a4d8d30b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:15:27.437147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a45e03a2-2364-4d89-960d-d0a2dbb4ef3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:27.437667 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a45e03a2-2364-4d89-960d-d0a2dbb4ef3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:27.443937 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1236/2462] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:15:27 2022] DELETE /identity/v3/projects/24fe3482c42347a3b533219a4d8d30b7 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:15:27.447207 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f1643-b641-4a04-b885-dd60b201b067 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:27.448926 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f1643-b641-4a04-b885-dd60b201b067 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:27.449310 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f1643-b641-4a04-b885-dd60b201b067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:27.477405 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-356f1643-b641-4a04-b885-dd60b201b067 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:27.479172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-356f1643-b641-4a04-b885-dd60b201b067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:27.530419 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:27.594316 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1227/2463] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:15:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:15:27.596523 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4483b7e-1c18-4f23-89dd-0b8088c5b1d7 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:27.597595 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4483b7e-1c18-4f23-89dd-0b8088c5b1d7 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:27.597981 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4483b7e-1c18-4f23-89dd-0b8088c5b1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:27.622577 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d4483b7e-1c18-4f23-89dd-0b8088c5b1d7 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:27.624188 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d4483b7e-1c18-4f23-89dd-0b8088c5b1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:27.636326 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1237/2464] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:15:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:15:27.648346 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f1643-b641-4a04-b885-dd60b201b067 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:27.660117 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f1643-b641-4a04-b885-dd60b201b067 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:27.669596 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd57cc1-b3ae-42d8-bb18-55737c1214f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:27.670309 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd57cc1-b3ae-42d8-bb18-55737c1214f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:27.670586 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd57cc1-b3ae-42d8-bb18-55737c1214f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:27.670940 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd57cc1-b3ae-42d8-bb18-55737c1214f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:27.671736 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd57cc1-b3ae-42d8-bb18-55737c1214f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:15:27.672101 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd57cc1-b3ae-42d8-bb18-55737c1214f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:27.674627 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd57cc1-b3ae-42d8-bb18-55737c1214f8 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:27.694692 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1228/2465] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:15:27.695900 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4483b7e-1c18-4f23-89dd-0b8088c5b1d7 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:27.723764 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4483b7e-1c18-4f23-89dd-0b8088c5b1d7 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:27.737801 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3e4ba5-de66-4b72-8ed6-a16ad31695f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:27.739703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e4ba5-de66-4b72-8ed6-a16ad31695f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:27.740385 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e4ba5-de66-4b72-8ed6-a16ad31695f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:27.742966 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e4ba5-de66-4b72-8ed6-a16ad31695f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:27.744209 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3e4ba5-de66-4b72-8ed6-a16ad31695f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:27.749316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3e4ba5-de66-4b72-8ed6-a16ad31695f7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:27.818602 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1238/2466] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:15:27 2022] POST /identity/v3/projects => generated 353 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:15:27.822702 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd57cc1-b3ae-42d8-bb18-55737c1214f8 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:27.839744 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd57cc1-b3ae-42d8-bb18-55737c1214f8 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:27.931197 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3774f9-af17-4311-b5a0-04a355e0004d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:27.932138 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3774f9-af17-4311-b5a0-04a355e0004d admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:27.932835 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3774f9-af17-4311-b5a0-04a355e0004d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:27.933166 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3774f9-af17-4311-b5a0-04a355e0004d admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:27.934017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3774f9-af17-4311-b5a0-04a355e0004d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:27.936226 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3774f9-af17-4311-b5a0-04a355e0004d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:27.983568 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1229/2467] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:15:27 2022] POST /identity/v3/users => generated 277 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:15:27.985095 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3e4ba5-de66-4b72-8ed6-a16ad31695f7 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:28.002725 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3e4ba5-de66-4b72-8ed6-a16ad31695f7 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:28.010733 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980f67e-6f5c-4b6c-97c4-824184f1dd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:28.011478 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980f67e-6f5c-4b6c-97c4-824184f1dd1c admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:28.011763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980f67e-6f5c-4b6c-97c4-824184f1dd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:28.012062 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980f67e-6f5c-4b6c-97c4-824184f1dd1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:28.012916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980f67e-6f5c-4b6c-97c4-824184f1dd1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:28.014658 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980f67e-6f5c-4b6c-97c4-824184f1dd1c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:28.027207 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1239/2468] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:15:28.032893 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3774f9-af17-4311-b5a0-04a355e0004d admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:28.045650 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3774f9-af17-4311-b5a0-04a355e0004d admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:28.055796 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b1047d-8b2c-44db-a35d-1e238d1db041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:28.056766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1047d-8b2c-44db-a35d-1e238d1db041 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:28.057544 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1047d-8b2c-44db-a35d-1e238d1db041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:28.057757 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1047d-8b2c-44db-a35d-1e238d1db041 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:28.059027 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b1047d-8b2c-44db-a35d-1e238d1db041 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:28.062065 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b1047d-8b2c-44db-a35d-1e238d1db041 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:28.088477 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1230/2469] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:15:28.091667 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980f67e-6f5c-4b6c-97c4-824184f1dd1c admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:28.104980 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980f67e-6f5c-4b6c-97c4-824184f1dd1c admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:28.113320 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7086d-386a-48e2-a11b-a859b6cb92a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:28.114478 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7086d-386a-48e2-a11b-a859b6cb92a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:28.115080 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7086d-386a-48e2-a11b-a859b6cb92a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:28.115566 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7086d-386a-48e2-a11b-a859b6cb92a7 admin admin] PATH_INFO: `/v3/projects/4d6aaa64040b408f9ac4f048c335de6f/users/17bab925b91d4dfb9434bfa5306c7212/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:28.140769 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e7086d-386a-48e2-a11b-a859b6cb92a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d6aaa64040b408f9ac4f048c335de6f, user_id=17bab925b91d4dfb9434bfa5306c7212, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:28.144154 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e7086d-386a-48e2-a11b-a859b6cb92a7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:28.188317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1240/2470] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:15:28 2022] PUT /identity/v3/projects/4d6aaa64040b408f9ac4f048c335de6f/users/17bab925b91d4dfb9434bfa5306c7212/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:15:28.196133 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea0af5-77d2-45e8-8ad1-dc1f8dd72d91 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:28.196678 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea0af5-77d2-45e8-8ad1-dc1f8dd72d91 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:28.197028 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea0af5-77d2-45e8-8ad1-dc1f8dd72d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:28.233605 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-64ea0af5-77d2-45e8-8ad1-dc1f8dd72d91 None None] MFA Rules not processed for user `17bab925b91d4dfb9434bfa5306c7212`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:28.235263 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-64ea0af5-77d2-45e8-8ad1-dc1f8dd72d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:28.348814 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1231/2471] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:15:28 2022] POST /identity/v3/auth/tokens => generated 3220 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:15:28.354337 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7086d-386a-48e2-a11b-a859b6cb92a7 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:28.441607 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7086d-386a-48e2-a11b-a859b6cb92a7 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:28.477589 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b32fe4-2dfa-4abf-b422-a65fe45a29d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:28.478804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b32fe4-2dfa-4abf-b422-a65fe45a29d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:28.479910 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b32fe4-2dfa-4abf-b422-a65fe45a29d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:28.479910 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b32fe4-2dfa-4abf-b422-a65fe45a29d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:28.480886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b32fe4-2dfa-4abf-b422-a65fe45a29d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:28.484047 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b32fe4-2dfa-4abf-b422-a65fe45a29d4 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:28.531625 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1241/2472] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:15:28 2022] POST /identity/v3/projects => generated 357 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:15:28.536675 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea0af5-77d2-45e8-8ad1-dc1f8dd72d91 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:28.556616 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea0af5-77d2-45e8-8ad1-dc1f8dd72d91 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:28.592416 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:28.637602 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf3aba-b9a3-427b-9472-c6429243fe90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:28.638466 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf3aba-b9a3-427b-9472-c6429243fe90 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:28.639064 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf3aba-b9a3-427b-9472-c6429243fe90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:28.639465 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf3aba-b9a3-427b-9472-c6429243fe90 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:28.640671 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf3aba-b9a3-427b-9472-c6429243fe90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:28.642663 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf3aba-b9a3-427b-9472-c6429243fe90 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:28.673864 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1232/2473] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:15:28 2022] POST /identity/v3/users => generated 285 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:15:28.676047 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b32fe4-2dfa-4abf-b422-a65fe45a29d4 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:28.693797 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b32fe4-2dfa-4abf-b422-a65fe45a29d4 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:28.700400 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb3eca-8504-4fc5-beaa-71ce2ce15951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:28.701318 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb3eca-8504-4fc5-beaa-71ce2ce15951 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:28.701870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb3eca-8504-4fc5-beaa-71ce2ce15951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:28.702335 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb3eca-8504-4fc5-beaa-71ce2ce15951 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:28.704008 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb3eca-8504-4fc5-beaa-71ce2ce15951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:28.705978 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb3eca-8504-4fc5-beaa-71ce2ce15951 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:28.719310 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1242/2474] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:15:28.726468 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf3aba-b9a3-427b-9472-c6429243fe90 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:28.737195 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf3aba-b9a3-427b-9472-c6429243fe90 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:28.742223 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afbf793-073c-4c49-98db-fd5f32c4cc15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:28.742995 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbf793-073c-4c49-98db-fd5f32c4cc15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:28.743279 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbf793-073c-4c49-98db-fd5f32c4cc15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:28.743620 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbf793-073c-4c49-98db-fd5f32c4cc15 admin admin] PATH_INFO: `/v3/projects/ec98b3488e5f482fac368424ba20f95d/users/47eab9d6a2da421d9becbb819690a935/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:28.761875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afbf793-073c-4c49-98db-fd5f32c4cc15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec98b3488e5f482fac368424ba20f95d, user_id=47eab9d6a2da421d9becbb819690a935, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:28.764661 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afbf793-073c-4c49-98db-fd5f32c4cc15 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:28.783345 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1233/2475] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:15:28 2022] PUT /identity/v3/projects/ec98b3488e5f482fac368424ba20f95d/users/47eab9d6a2da421d9becbb819690a935/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:15:28.787580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb3eca-8504-4fc5-beaa-71ce2ce15951 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:28.801056 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb3eca-8504-4fc5-beaa-71ce2ce15951 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:28.883266 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:28.909002 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f4c26f-1dd7-4bcd-a574-7bbf6379c2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:28.909907 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4c26f-1dd7-4bcd-a574-7bbf6379c2e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:28.910356 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4c26f-1dd7-4bcd-a574-7bbf6379c2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:28.910801 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4c26f-1dd7-4bcd-a574-7bbf6379c2e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:28.912055 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f4c26f-1dd7-4bcd-a574-7bbf6379c2e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:28.915624 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f4c26f-1dd7-4bcd-a574-7bbf6379c2e6 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:28.930136 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1243/2476] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:15:28.935235 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afbf793-073c-4c49-98db-fd5f32c4cc15 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:28.952421 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afbf793-073c-4c49-98db-fd5f32c4cc15 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:28.963932 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fcf59d-a316-4a81-87fe-c76083d268e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:28.964838 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fcf59d-a316-4a81-87fe-c76083d268e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:28.965296 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fcf59d-a316-4a81-87fe-c76083d268e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:28.965771 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fcf59d-a316-4a81-87fe-c76083d268e1 admin admin] PATH_INFO: `/v3/projects/ec98b3488e5f482fac368424ba20f95d/users/47eab9d6a2da421d9becbb819690a935/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:28.968880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fcf59d-a316-4a81-87fe-c76083d268e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec98b3488e5f482fac368424ba20f95d, user_id=47eab9d6a2da421d9becbb819690a935, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:28.972177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fcf59d-a316-4a81-87fe-c76083d268e1 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:28.985557 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1234/2477] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:15:28 2022] PUT /identity/v3/projects/ec98b3488e5f482fac368424ba20f95d/users/47eab9d6a2da421d9becbb819690a935/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:15:28.996605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d6813-4234-4ae7-9dd2-cd4885bb32f2 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:28.996875 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d6813-4234-4ae7-9dd2-cd4885bb32f2 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:28.997201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d6813-4234-4ae7-9dd2-cd4885bb32f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:29.050339 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-446d6813-4234-4ae7-9dd2-cd4885bb32f2 None None] MFA Rules not processed for user `47eab9d6a2da421d9becbb819690a935`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:29.053227 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-446d6813-4234-4ae7-9dd2-cd4885bb32f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:29.177250 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1244/2478] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:15:28 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:15:29.201516 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fcf59d-a316-4a81-87fe-c76083d268e1 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:29.204055 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-68fcf59d-a316-4a81-87fe-c76083d268e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:29.305856 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fcf59d-a316-4a81-87fe-c76083d268e1 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:29.310541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9eacf-510b-4b99-9499-541f0287ecc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:29.311265 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9eacf-510b-4b99-9499-541f0287ecc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:29.311613 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9eacf-510b-4b99-9499-541f0287ecc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:29.312012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9eacf-510b-4b99-9499-541f0287ecc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:29.324655 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9eacf-510b-4b99-9499-541f0287ecc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:29.326180 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9eacf-510b-4b99-9499-541f0287ecc5 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:29.352198 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1235/2479] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:15:29 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:15:29.445331 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d6813-4234-4ae7-9dd2-cd4885bb32f2 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:29.459789 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d6813-4234-4ae7-9dd2-cd4885bb32f2 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:29.465497 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aee18c-4e00-429d-afbe-2f5d4f38b710 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:29.466242 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aee18c-4e00-429d-afbe-2f5d4f38b710 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:29.467147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aee18c-4e00-429d-afbe-2f5d4f38b710 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:29.467147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aee18c-4e00-429d-afbe-2f5d4f38b710 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:29.552987 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aee18c-4e00-429d-afbe-2f5d4f38b710 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:29.554517 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aee18c-4e00-429d-afbe-2f5d4f38b710 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:29.572877 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1245/2480] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:15:29 2022] GET /identity/v3/auth/tokens => generated 3220 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:15:32.945424 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c3eec-1082-4089-aec3-2de144c82452 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:32.945906 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c3eec-1082-4089-aec3-2de144c82452 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:32.946013 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c3eec-1082-4089-aec3-2de144c82452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:32.966560 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-3e6c3eec-1082-4089-aec3-2de144c82452 None None] MFA Rules not processed for user `e07e83c0bdba47d39c09308eba1bdaae`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:32.967788 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3e6c3eec-1082-4089-aec3-2de144c82452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:33.050879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1236/2481] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Aug 1 14:15:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:15:33.104239 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aee18c-4e00-429d-afbe-2f5d4f38b710 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:33.105729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-53aee18c-4e00-429d-afbe-2f5d4f38b710 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:33.200982 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aee18c-4e00-429d-afbe-2f5d4f38b710 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:33.206687 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13dd34-0410-4ea5-b933-0c4210cef9e5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cda7f365ba0405daaf86870960f12d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:33.207342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13dd34-0410-4ea5-b933-0c4210cef9e5 service swift] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:33.207622 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13dd34-0410-4ea5-b933-0c4210cef9e5 service swift] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:33.207897 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13dd34-0410-4ea5-b933-0c4210cef9e5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:33.218250 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e13dd34-0410-4ea5-b933-0c4210cef9e5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:33.219867 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e13dd34-0410-4ea5-b933-0c4210cef9e5 service swift] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:33.230156 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1246/2482] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 14:15:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:15:43.043047 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=82,keystone:INSERT=5 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:43.225690 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:46.014389 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c3eec-1082-4089-aec3-2de144c82452 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:46.091782 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c3eec-1082-4089-aec3-2de144c82452 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:46.097598 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c86a2-19d4-498c-a0d0-deaa9f3b3ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:46.098264 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c86a2-19d4-498c-a0d0-deaa9f3b3ad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:46.098542 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c86a2-19d4-498c-a0d0-deaa9f3b3ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:46.098809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c86a2-19d4-498c-a0d0-deaa9f3b3ad1 admin admin] PATH_INFO: `/v3/users/17bab925b91d4dfb9434bfa5306c7212` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:46.100139 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988c86a2-19d4-498c-a0d0-deaa9f3b3ad1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17bab925b91d4dfb9434bfa5306c7212)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:46.102167 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988c86a2-19d4-498c-a0d0-deaa9f3b3ad1 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:46.153525 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-988c86a2-19d4-498c-a0d0-deaa9f3b3ad1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17bab925b91d4dfb9434bfa5306c7212'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:46.154187 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-988c86a2-19d4-498c-a0d0-deaa9f3b3ad1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17bab925b91d4dfb9434bfa5306c7212'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:46.162395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-988c86a2-19d4-498c-a0d0-deaa9f3b3ad1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17bab925b91d4dfb9434bfa5306c7212'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:46.170460 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-988c86a2-19d4-498c-a0d0-deaa9f3b3ad1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17bab925b91d4dfb9434bfa5306c7212'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:46.183615 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-988c86a2-19d4-498c-a0d0-deaa9f3b3ad1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17bab925b91d4dfb9434bfa5306c7212'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:46.190496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1237/2483] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:15:46 2022] DELETE /identity/v3/users/17bab925b91d4dfb9434bfa5306c7212 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:15:46.195762 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a4c72-75da-47ee-b095-1b935df8a24d None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:46.196302 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a4c72-75da-47ee-b095-1b935df8a24d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:46.196403 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a4c72-75da-47ee-b095-1b935df8a24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:46.220690 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-0c0a4c72-75da-47ee-b095-1b935df8a24d None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:46.221929 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0c0a4c72-75da-47ee-b095-1b935df8a24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:46.287892 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1247/2484] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:15:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:15:46.299003 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c86a2-19d4-498c-a0d0-deaa9f3b3ad1 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:46.301096 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-988c86a2-19d4-498c-a0d0-deaa9f3b3ad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:46.362765 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c86a2-19d4-498c-a0d0-deaa9f3b3ad1 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:46.366654 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f54e931-855e-47a5-892f-0e9bfe58b9d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:46.367322 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54e931-855e-47a5-892f-0e9bfe58b9d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:46.367596 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54e931-855e-47a5-892f-0e9bfe58b9d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:46.367864 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54e931-855e-47a5-892f-0e9bfe58b9d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:46.377651 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f54e931-855e-47a5-892f-0e9bfe58b9d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:46.379120 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f54e931-855e-47a5-892f-0e9bfe58b9d1 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:46.389342 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1238/2485] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:15:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:15:46.762558 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a4c72-75da-47ee-b095-1b935df8a24d None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:46.764752 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0c0a4c72-75da-47ee-b095-1b935df8a24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:46.779203 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a4c72-75da-47ee-b095-1b935df8a24d None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:46.785576 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7aa87-338e-4292-bca8-4b4065fe68e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:46.786356 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7aa87-338e-4292-bca8-4b4065fe68e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:46.786636 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7aa87-338e-4292-bca8-4b4065fe68e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:46.786908 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7aa87-338e-4292-bca8-4b4065fe68e7 admin admin] PATH_INFO: `/v3/projects/4d6aaa64040b408f9ac4f048c335de6f` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:46.788231 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7aa87-338e-4292-bca8-4b4065fe68e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d6aaa64040b408f9ac4f048c335de6f)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:46.790630 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7aa87-338e-4292-bca8-4b4065fe68e7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:46.856605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-efe7aa87-338e-4292-bca8-4b4065fe68e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d6aaa64040b408f9ac4f048c335de6f', 'request_id': 'req-efe7aa87-338e-4292-bca8-4b4065fe68e7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:46.863899 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-efe7aa87-338e-4292-bca8-4b4065fe68e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d6aaa64040b408f9ac4f048c335de6f', 'request_id': 'req-efe7aa87-338e-4292-bca8-4b4065fe68e7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:46.878750 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-efe7aa87-338e-4292-bca8-4b4065fe68e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d6aaa64040b408f9ac4f048c335de6f', 'request_id': 'req-efe7aa87-338e-4292-bca8-4b4065fe68e7'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:46.906324 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-efe7aa87-338e-4292-bca8-4b4065fe68e7 admin admin] The token cache is being invalidate because project 4d6aaa64040b408f9ac4f048c335de6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:15:46.906612 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-efe7aa87-338e-4292-bca8-4b4065fe68e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:46.907398 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-efe7aa87-338e-4292-bca8-4b4065fe68e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:46.909402 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1248/2486] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:15:46 2022] DELETE /identity/v3/projects/4d6aaa64040b408f9ac4f048c335de6f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:15:46.913561 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f54e931-855e-47a5-892f-0e9bfe58b9d1 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:46.932721 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f54e931-855e-47a5-892f-0e9bfe58b9d1 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:46.960556 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:47.024145 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc1dbe-4ba9-4e73-bc05-51ba7cfd36b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:47.024876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc1dbe-4ba9-4e73-bc05-51ba7cfd36b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:47.025115 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc1dbe-4ba9-4e73-bc05-51ba7cfd36b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:47.025360 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc1dbe-4ba9-4e73-bc05-51ba7cfd36b7 admin admin] PATH_INFO: `/v3/users/47eab9d6a2da421d9becbb819690a935` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:47.026780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc1dbe-4ba9-4e73-bc05-51ba7cfd36b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47eab9d6a2da421d9becbb819690a935)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:47.029222 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc1dbe-4ba9-4e73-bc05-51ba7cfd36b7 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:47.103127 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e5bc1dbe-4ba9-4e73-bc05-51ba7cfd36b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47eab9d6a2da421d9becbb819690a935'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:47.104055 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e5bc1dbe-4ba9-4e73-bc05-51ba7cfd36b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47eab9d6a2da421d9becbb819690a935'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:47.114054 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e5bc1dbe-4ba9-4e73-bc05-51ba7cfd36b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47eab9d6a2da421d9becbb819690a935'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:47.121923 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e5bc1dbe-4ba9-4e73-bc05-51ba7cfd36b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47eab9d6a2da421d9becbb819690a935'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:47.135945 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e5bc1dbe-4ba9-4e73-bc05-51ba7cfd36b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47eab9d6a2da421d9becbb819690a935'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:47.145001 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1239/2487] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:15:46 2022] DELETE /identity/v3/users/47eab9d6a2da421d9becbb819690a935 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:15:47.764650 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7aa87-338e-4292-bca8-4b4065fe68e7 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:47.766853 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-efe7aa87-338e-4292-bca8-4b4065fe68e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:47.783414 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7aa87-338e-4292-bca8-4b4065fe68e7 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:47.855891 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cb776-3b2d-420c-9b08-f086e5932929 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:47.856184 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cb776-3b2d-420c-9b08-f086e5932929 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:47.856487 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cb776-3b2d-420c-9b08-f086e5932929 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:47.858010 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1240/2488] 10.222.0.45 () {54 vars in 921 bytes} [Mon Aug 1 14:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:15:47.866337 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a20f9-0c96-4ba4-8977-eaafcb2d87a8 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:47.866636 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a20f9-0c96-4ba4-8977-eaafcb2d87a8 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:47.866879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a20f9-0c96-4ba4-8977-eaafcb2d87a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:47.889427 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9197883f-369c-4142-8629-6b9c89a15cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:47.890100 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197883f-369c-4142-8629-6b9c89a15cf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:47.890405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197883f-369c-4142-8629-6b9c89a15cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:47.890684 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197883f-369c-4142-8629-6b9c89a15cf1 admin admin] PATH_INFO: `/v3/projects/ec98b3488e5f482fac368424ba20f95d` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:47.890870 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-a46a20f9-0c96-4ba4-8977-eaafcb2d87a8 None None] MFA Rules not processed for user `5b3adc14dcaf4ac79b43bb97fa1ba1d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:47.891906 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9197883f-369c-4142-8629-6b9c89a15cf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec98b3488e5f482fac368424ba20f95d)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:47.892576 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a46a20f9-0c96-4ba4-8977-eaafcb2d87a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:47.893906 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9197883f-369c-4142-8629-6b9c89a15cf1 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:47.931217 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:47.971112 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-9197883f-369c-4142-8629-6b9c89a15cf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec98b3488e5f482fac368424ba20f95d', 'request_id': 'req-9197883f-369c-4142-8629-6b9c89a15cf1'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:47.983262 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-9197883f-369c-4142-8629-6b9c89a15cf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec98b3488e5f482fac368424ba20f95d', 'request_id': 'req-9197883f-369c-4142-8629-6b9c89a15cf1'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:47.999301 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-9197883f-369c-4142-8629-6b9c89a15cf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec98b3488e5f482fac368424ba20f95d', 'request_id': 'req-9197883f-369c-4142-8629-6b9c89a15cf1'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:48.003688 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1241/2489] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Aug 1 14:15:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:15:48.015656 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-9197883f-369c-4142-8629-6b9c89a15cf1 admin admin] The token cache is being invalidate because project ec98b3488e5f482fac368424ba20f95d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:15:48.015949 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-9197883f-369c-4142-8629-6b9c89a15cf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:48.016679 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-9197883f-369c-4142-8629-6b9c89a15cf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:48.018341 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1249/2490] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:15:47 2022] DELETE /identity/v3/projects/ec98b3488e5f482fac368424ba20f95d => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:15:48.023427 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25561300-c94f-4c41-b649-f3c6fd75002d None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:48.023715 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25561300-c94f-4c41-b649-f3c6fd75002d None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:48.023994 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25561300-c94f-4c41-b649-f3c6fd75002d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:48.031386 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9197883f-369c-4142-8629-6b9c89a15cf1 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:48.033645 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9197883f-369c-4142-8629-6b9c89a15cf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:48.046217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-25561300-c94f-4c41-b649-f3c6fd75002d None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:48.047468 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-25561300-c94f-4c41-b649-f3c6fd75002d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:48.088248 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:48.100451 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9197883f-369c-4142-8629-6b9c89a15cf1 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:48.104963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b28798f-debc-4957-bcec-bd951b458f5f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7317c8829cb4fa1b30711f1db575cab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:48.105617 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28798f-debc-4957-bcec-bd951b458f5f service glance] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:48.105897 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28798f-debc-4957-bcec-bd951b458f5f service glance] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:48.106174 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28798f-debc-4957-bcec-bd951b458f5f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:48.108062 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4b28798f-debc-4957-bcec-bd951b458f5f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:48.126132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1242/2491] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:15:48.134976 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd68610-a14c-4d17-b377-09173d18c605 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:48.135294 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd68610-a14c-4d17-b377-09173d18c605 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:48.135544 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd68610-a14c-4d17-b377-09173d18c605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:48.157408 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-bbd68610-a14c-4d17-b377-09173d18c605 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:48.158657 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-bbd68610-a14c-4d17-b377-09173d18c605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:48.167797 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1243/2492] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:15:48.171977 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd68610-a14c-4d17-b377-09173d18c605 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:48.183043 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd68610-a14c-4d17-b377-09173d18c605 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:48.188846 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f965203c-98ba-4ccd-a133-f6112fd1234e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:48.189863 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965203c-98ba-4ccd-a133-f6112fd1234e admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:48.190286 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965203c-98ba-4ccd-a133-f6112fd1234e admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:48.190583 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965203c-98ba-4ccd-a133-f6112fd1234e admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:48.191532 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f965203c-98ba-4ccd-a133-f6112fd1234e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:15:48.191831 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f965203c-98ba-4ccd-a133-f6112fd1234e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:48.194236 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f965203c-98ba-4ccd-a133-f6112fd1234e admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:48.207289 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1244/2493] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:15:48.208662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b28798f-debc-4957-bcec-bd951b458f5f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:48.210242 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b28798f-debc-4957-bcec-bd951b458f5f service glance] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:48.212214 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f965203c-98ba-4ccd-a133-f6112fd1234e admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:48.227879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f965203c-98ba-4ccd-a133-f6112fd1234e admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:48.228142 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1250/2494] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:15:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:15:48.228247 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd0ccb-2978-4acc-b1c3-bfe5846d7059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:48.228247 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd0ccb-2978-4acc-b1c3-bfe5846d7059 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:48.228458 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd0ccb-2978-4acc-b1c3-bfe5846d7059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:48.228687 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd0ccb-2978-4acc-b1c3-bfe5846d7059 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:48.229353 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd0ccb-2978-4acc-b1c3-bfe5846d7059 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:48.231402 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd0ccb-2978-4acc-b1c3-bfe5846d7059 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:48.262382 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1245/2495] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:15:48 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:15:48.266581 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b28798f-debc-4957-bcec-bd951b458f5f service glance] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:48.277729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b28798f-debc-4957-bcec-bd951b458f5f service glance] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:48.303121 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c22dd7e-4e5a-4ce9-86b0-04b998e2d788 None None] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:48.303538 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c22dd7e-4e5a-4ce9-86b0-04b998e2d788 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:48.303880 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c22dd7e-4e5a-4ce9-86b0-04b998e2d788 None None] PATH_INFO: `/` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:48.306118 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1246/2496] 10.222.0.45 () {54 vars in 921 bytes} [Mon Aug 1 14:15:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:15:48.310384 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d31a10-285d-4e18-99cd-25d62c9dde72 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:48.310675 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d31a10-285d-4e18-99cd-25d62c9dde72 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:48.310922 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d31a10-285d-4e18-99cd-25d62c9dde72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:48.332716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-47d31a10-285d-4e18-99cd-25d62c9dde72 None None] MFA Rules not processed for user `6dd44b7a78594288b6aabb393fd6324a`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:48.334297 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-47d31a10-285d-4e18-99cd-25d62c9dde72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:48.342720 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:48.357215 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee227260-f599-423b-85d6-e47e4996e226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:48.358162 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee227260-f599-423b-85d6-e47e4996e226 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:48.358700 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee227260-f599-423b-85d6-e47e4996e226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:48.359158 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee227260-f599-423b-85d6-e47e4996e226 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:48.360021 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee227260-f599-423b-85d6-e47e4996e226 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:48.362370 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee227260-f599-423b-85d6-e47e4996e226 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:48.388039 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1251/2497] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:15:48 2022] POST /identity/v3/users => generated 294 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:15:48.392259 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee227260-f599-423b-85d6-e47e4996e226 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:48.402522 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee227260-f599-423b-85d6-e47e4996e226 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:48.405714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1247/2498] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Aug 1 14:15:48 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:15:48.408137 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc0e554-7ea4-4240-9a92-5d457831d822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:48.408805 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc0e554-7ea4-4240-9a92-5d457831d822 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:48.409083 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc0e554-7ea4-4240-9a92-5d457831d822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:48.409351 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc0e554-7ea4-4240-9a92-5d457831d822 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:48.410059 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc0e554-7ea4-4240-9a92-5d457831d822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:48.411946 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc0e554-7ea4-4240-9a92-5d457831d822 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:48.419383 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d31a10-285d-4e18-99cd-25d62c9dde72 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:48.421098 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-47d31a10-285d-4e18-99cd-25d62c9dde72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:48.426546 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1252/2499] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:15:48.431155 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc0e554-7ea4-4240-9a92-5d457831d822 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:48.433989 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d31a10-285d-4e18-99cd-25d62c9dde72 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:48.439505 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9349db4-c907-41d7-a7e7-93616417858f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:48.440137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9349db4-c907-41d7-a7e7-93616417858f service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:48.440441 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9349db4-c907-41d7-a7e7-93616417858f service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:48.440711 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9349db4-c907-41d7-a7e7-93616417858f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:48.442237 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc0e554-7ea4-4240-9a92-5d457831d822 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:48.447046 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bcb4ea-379e-488f-8820-a54d93c30177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:48.447669 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bcb4ea-379e-488f-8820-a54d93c30177 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:48.447936 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bcb4ea-379e-488f-8820-a54d93c30177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:48.448205 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bcb4ea-379e-488f-8820-a54d93c30177 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:48.449046 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bcb4ea-379e-488f-8820-a54d93c30177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:48.450098 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9349db4-c907-41d7-a7e7-93616417858f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:48.450665 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bcb4ea-379e-488f-8820-a54d93c30177 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:48.453643 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9349db4-c907-41d7-a7e7-93616417858f service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:48.467551 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1253/2500] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:15:48.467768 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1248/2501] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:15:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:15:48.468693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bcb4ea-379e-488f-8820-a54d93c30177 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:48.479814 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bcb4ea-379e-488f-8820-a54d93c30177 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:48.485138 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b59dd-bb2d-4aaf-afa3-3b5b87499585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:48.485816 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b59dd-bb2d-4aaf-afa3-3b5b87499585 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:48.486243 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b59dd-bb2d-4aaf-afa3-3b5b87499585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:48.486517 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b59dd-bb2d-4aaf-afa3-3b5b87499585 admin admin] PATH_INFO: `/v3/projects/c3d0451c58d14a098c16be6cf7201cf0/users/42afe26e5dc14b5fab485d450375d1d6/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:48.508512 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b59dd-bb2d-4aaf-afa3-3b5b87499585 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3d0451c58d14a098c16be6cf7201cf0, user_id=42afe26e5dc14b5fab485d450375d1d6, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:48.511196 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b59dd-bb2d-4aaf-afa3-3b5b87499585 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:48.528653 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1254/2502] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:15:48 2022] PUT /identity/v3/projects/c3d0451c58d14a098c16be6cf7201cf0/users/42afe26e5dc14b5fab485d450375d1d6/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:15:48.539204 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b79354-356a-4fc1-a83c-9720e4e48c90 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:48.539978 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b79354-356a-4fc1-a83c-9720e4e48c90 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:48.540536 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b79354-356a-4fc1-a83c-9720e4e48c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:48.557296 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feff6ef8-7cfa-46e3-92b7-94492f97062d None None] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:48.557557 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feff6ef8-7cfa-46e3-92b7-94492f97062d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:48.557905 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feff6ef8-7cfa-46e3-92b7-94492f97062d None None] PATH_INFO: `/` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:48.559205 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1255/2503] 10.222.0.45 () {54 vars in 921 bytes} [Mon Aug 1 14:15:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:15:48.565241 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3c604-94c6-4f4d-a961-6c0cb9676fe4 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:48.565515 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3c604-94c6-4f4d-a961-6c0cb9676fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:48.565697 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3c604-94c6-4f4d-a961-6c0cb9676fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:48.574979 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-53b79354-356a-4fc1-a83c-9720e4e48c90 None None] MFA Rules not processed for user `42afe26e5dc14b5fab485d450375d1d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:48.576210 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-53b79354-356a-4fc1-a83c-9720e4e48c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:48.587129 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-50c3c604-94c6-4f4d-a961-6c0cb9676fe4 None None] MFA Rules not processed for user `5b3adc14dcaf4ac79b43bb97fa1ba1d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:48.588504 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-50c3c604-94c6-4f4d-a961-6c0cb9676fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:48.649223 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1249/2504] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:15:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:15:48.654311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b79354-356a-4fc1-a83c-9720e4e48c90 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:48.661616 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1256/2505] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Aug 1 14:15:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:15:48.673178 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c3c604-94c6-4f4d-a961-6c0cb9676fe4 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:48.675988 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:48.731850 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b79354-356a-4fc1-a83c-9720e4e48c90 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:48.737053 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d01972b-c33c-4bc6-a744-836f9caf0cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:48.737619 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d01972b-c33c-4bc6-a744-836f9caf0cb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:48.737939 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d01972b-c33c-4bc6-a744-836f9caf0cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:48.738339 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d01972b-c33c-4bc6-a744-836f9caf0cb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:48.738539 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c3c604-94c6-4f4d-a961-6c0cb9676fe4 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:48.739023 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d01972b-c33c-4bc6-a744-836f9caf0cb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:48.741018 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d01972b-c33c-4bc6-a744-836f9caf0cb0 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:48.743069 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6feb57-8525-4ab1-8376-661c48d01812 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:48.743679 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6feb57-8525-4ab1-8376-661c48d01812 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:48.743954 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6feb57-8525-4ab1-8376-661c48d01812 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:48.744221 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6feb57-8525-4ab1-8376-661c48d01812 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:48.755389 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6feb57-8525-4ab1-8376-661c48d01812 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:48.756889 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6feb57-8525-4ab1-8376-661c48d01812 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:48.769784 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1257/2506] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:15:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:15:48.772592 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1250/2507] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:15:48 2022] POST /identity/v3/projects => generated 387 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:15:48.782338 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6feb57-8525-4ab1-8376-661c48d01812 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:48.793591 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6feb57-8525-4ab1-8376-661c48d01812 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:48.827173 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:48.871315 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d465c-020e-4188-9c3a-17c61b3e112d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:48.872379 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d465c-020e-4188-9c3a-17c61b3e112d admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:48.872823 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d465c-020e-4188-9c3a-17c61b3e112d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:48.873270 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d465c-020e-4188-9c3a-17c61b3e112d admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:48.874400 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17d465c-020e-4188-9c3a-17c61b3e112d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:48.877890 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17d465c-020e-4188-9c3a-17c61b3e112d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:48.909407 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1258/2508] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:15:48 2022] POST /identity/v3/users => generated 300 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:15:48.913708 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d01972b-c33c-4bc6-a744-836f9caf0cb0 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:48.926421 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d01972b-c33c-4bc6-a744-836f9caf0cb0 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:48.932624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1edc69a-1e0a-4ced-975b-ee9e971015a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:48.933365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1edc69a-1e0a-4ced-975b-ee9e971015a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:48.933735 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1edc69a-1e0a-4ced-975b-ee9e971015a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:48.934075 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1edc69a-1e0a-4ced-975b-ee9e971015a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:48.934874 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1edc69a-1e0a-4ced-975b-ee9e971015a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:48.937263 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1edc69a-1e0a-4ced-975b-ee9e971015a2 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:48.953970 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1251/2509] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:15:48.955267 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d465c-020e-4188-9c3a-17c61b3e112d admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:48.966861 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d465c-020e-4188-9c3a-17c61b3e112d admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:48.972634 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6519ae7-5bcd-4b72-9a5e-77144b73e028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:48.973354 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6519ae7-5bcd-4b72-9a5e-77144b73e028 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:48.973703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6519ae7-5bcd-4b72-9a5e-77144b73e028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:48.974038 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6519ae7-5bcd-4b72-9a5e-77144b73e028 admin admin] PATH_INFO: `/v3/projects/b7ebace868cb4f5fafa8531836e754bc/users/f917f91eeb1644acb46f8a734cdf73e0/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:48.997325 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6519ae7-5bcd-4b72-9a5e-77144b73e028 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7ebace868cb4f5fafa8531836e754bc, user_id=f917f91eeb1644acb46f8a734cdf73e0, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:49.000694 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6519ae7-5bcd-4b72-9a5e-77144b73e028 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:49.026024 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1259/2510] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:15:48 2022] PUT /identity/v3/projects/b7ebace868cb4f5fafa8531836e754bc/users/f917f91eeb1644acb46f8a734cdf73e0/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:15:49.030412 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1edc69a-1e0a-4ced-975b-ee9e971015a2 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:49.043756 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1edc69a-1e0a-4ced-975b-ee9e971015a2 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:49.124590 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c088d5-624b-48c1-83a4-29e7ab2a162b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:49.125430 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c088d5-624b-48c1-83a4-29e7ab2a162b admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:49.125774 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c088d5-624b-48c1-83a4-29e7ab2a162b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:49.126110 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c088d5-624b-48c1-83a4-29e7ab2a162b admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:49.126888 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c088d5-624b-48c1-83a4-29e7ab2a162b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:49.128695 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c088d5-624b-48c1-83a4-29e7ab2a162b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:49.140182 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1252/2511] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:15:49.144407 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6519ae7-5bcd-4b72-9a5e-77144b73e028 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:49.155978 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6519ae7-5bcd-4b72-9a5e-77144b73e028 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:49.160658 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf6cc1-bdf1-4afb-918c-8c50921d2ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:49.161627 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf6cc1-bdf1-4afb-918c-8c50921d2ee3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:49.161627 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf6cc1-bdf1-4afb-918c-8c50921d2ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:49.162082 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf6cc1-bdf1-4afb-918c-8c50921d2ee3 admin admin] PATH_INFO: `/v3/projects/b7ebace868cb4f5fafa8531836e754bc/users/f917f91eeb1644acb46f8a734cdf73e0/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:49.164892 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf6cc1-bdf1-4afb-918c-8c50921d2ee3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7ebace868cb4f5fafa8531836e754bc, user_id=f917f91eeb1644acb46f8a734cdf73e0, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:49.167493 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf6cc1-bdf1-4afb-918c-8c50921d2ee3 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:49.180686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1260/2512] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:15:49 2022] PUT /identity/v3/projects/b7ebace868cb4f5fafa8531836e754bc/users/f917f91eeb1644acb46f8a734cdf73e0/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:15:49.192368 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1f888-7d59-42c6-8dd3-e096a0193a9d None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:49.192677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1f888-7d59-42c6-8dd3-e096a0193a9d None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:49.193003 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1f888-7d59-42c6-8dd3-e096a0193a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:49.229251 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-7aa1f888-7d59-42c6-8dd3-e096a0193a9d None None] MFA Rules not processed for user `f917f91eeb1644acb46f8a734cdf73e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:49.230867 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7aa1f888-7d59-42c6-8dd3-e096a0193a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:49.307977 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1253/2513] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:15:49 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:15:49.319739 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf6cc1-bdf1-4afb-918c-8c50921d2ee3 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:49.321719 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3daf6cc1-bdf1-4afb-918c-8c50921d2ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:49.385475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf6cc1-bdf1-4afb-918c-8c50921d2ee3 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:49.389297 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf46d5-b7a3-46a3-81e0-136a6baedde9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:49.389840 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf46d5-b7a3-46a3-81e0-136a6baedde9 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:49.390114 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf46d5-b7a3-46a3-81e0-136a6baedde9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:49.390383 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf46d5-b7a3-46a3-81e0-136a6baedde9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:49.400725 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecf46d5-b7a3-46a3-81e0-136a6baedde9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:49.402191 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecf46d5-b7a3-46a3-81e0-136a6baedde9 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:49.414785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1261/2514] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:15:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:15:49.446099 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1f888-7d59-42c6-8dd3-e096a0193a9d None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:49.465367 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:49.523637 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1f888-7d59-42c6-8dd3-e096a0193a9d None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:49.529653 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0df08f-e3de-4840-b6cb-f82c3abecf06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:49.530358 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0df08f-e3de-4840-b6cb-f82c3abecf06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:49.530631 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0df08f-e3de-4840-b6cb-f82c3abecf06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:49.530879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0df08f-e3de-4840-b6cb-f82c3abecf06 admin admin] PATH_INFO: `/v3/users/42afe26e5dc14b5fab485d450375d1d6` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:49.532212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0df08f-e3de-4840-b6cb-f82c3abecf06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42afe26e5dc14b5fab485d450375d1d6)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:49.534671 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0df08f-e3de-4840-b6cb-f82c3abecf06 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:49.586434 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-6d0df08f-e3de-4840-b6cb-f82c3abecf06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42afe26e5dc14b5fab485d450375d1d6'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:49.587071 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-6d0df08f-e3de-4840-b6cb-f82c3abecf06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42afe26e5dc14b5fab485d450375d1d6'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:49.595819 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-6d0df08f-e3de-4840-b6cb-f82c3abecf06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42afe26e5dc14b5fab485d450375d1d6'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:49.603908 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-6d0df08f-e3de-4840-b6cb-f82c3abecf06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42afe26e5dc14b5fab485d450375d1d6'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:49.617724 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-6d0df08f-e3de-4840-b6cb-f82c3abecf06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42afe26e5dc14b5fab485d450375d1d6'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:49.623999 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1254/2515] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:15:49 2022] DELETE /identity/v3/users/42afe26e5dc14b5fab485d450375d1d6 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:15:49.629096 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574fe76-ab80-4c4d-bf57-0e21ff3b8e56 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:49.629372 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574fe76-ab80-4c4d-bf57-0e21ff3b8e56 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:49.629638 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574fe76-ab80-4c4d-bf57-0e21ff3b8e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:49.648639 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a574fe76-ab80-4c4d-bf57-0e21ff3b8e56 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:49.649832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a574fe76-ab80-4c4d-bf57-0e21ff3b8e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:49.716451 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1262/2516] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:15:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:15:49.730565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0df08f-e3de-4840-b6cb-f82c3abecf06 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:49.733155 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-6d0df08f-e3de-4840-b6cb-f82c3abecf06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:49.806151 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0df08f-e3de-4840-b6cb-f82c3abecf06 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:49.809976 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce518f1a-6ac0-4637-be91-32088c73105e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:49.810615 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce518f1a-6ac0-4637-be91-32088c73105e service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:49.810919 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce518f1a-6ac0-4637-be91-32088c73105e service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:49.811187 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce518f1a-6ac0-4637-be91-32088c73105e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:49.822028 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce518f1a-6ac0-4637-be91-32088c73105e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:49.823757 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce518f1a-6ac0-4637-be91-32088c73105e service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:49.835085 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1255/2517] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:15:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:15:50.142847 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574fe76-ab80-4c4d-bf57-0e21ff3b8e56 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:50.144387 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a574fe76-ab80-4c4d-bf57-0e21ff3b8e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:50.157949 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574fe76-ab80-4c4d-bf57-0e21ff3b8e56 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:50.165225 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e8001-bb35-4931-91f7-daf510655dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:50.165957 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e8001-bb35-4931-91f7-daf510655dab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:50.166244 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e8001-bb35-4931-91f7-daf510655dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:50.166656 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e8001-bb35-4931-91f7-daf510655dab admin admin] PATH_INFO: `/v3/projects/c3d0451c58d14a098c16be6cf7201cf0` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:50.167956 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602e8001-bb35-4931-91f7-daf510655dab admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3d0451c58d14a098c16be6cf7201cf0)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:50.170239 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602e8001-bb35-4931-91f7-daf510655dab admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:50.200182 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:50.227521 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-602e8001-bb35-4931-91f7-daf510655dab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3d0451c58d14a098c16be6cf7201cf0', 'request_id': 'req-602e8001-bb35-4931-91f7-daf510655dab'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:50.235018 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-602e8001-bb35-4931-91f7-daf510655dab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3d0451c58d14a098c16be6cf7201cf0', 'request_id': 'req-602e8001-bb35-4931-91f7-daf510655dab'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:50.247490 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-602e8001-bb35-4931-91f7-daf510655dab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3d0451c58d14a098c16be6cf7201cf0', 'request_id': 'req-602e8001-bb35-4931-91f7-daf510655dab'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:50.263814 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-602e8001-bb35-4931-91f7-daf510655dab admin admin] The token cache is being invalidate because project c3d0451c58d14a098c16be6cf7201cf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:15:50.264044 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-602e8001-bb35-4931-91f7-daf510655dab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:50.264752 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-602e8001-bb35-4931-91f7-daf510655dab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:50.269435 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1263/2518] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:15:50 2022] DELETE /identity/v3/projects/c3d0451c58d14a098c16be6cf7201cf0 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:15:50.270564 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce518f1a-6ac0-4637-be91-32088c73105e service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:50.282473 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce518f1a-6ac0-4637-be91-32088c73105e service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:50.326841 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:50.363320 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaec7ed-45e7-452c-b114-4278e822e801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:50.363985 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaec7ed-45e7-452c-b114-4278e822e801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:50.364401 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaec7ed-45e7-452c-b114-4278e822e801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:50.364600 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaec7ed-45e7-452c-b114-4278e822e801 admin admin] PATH_INFO: `/v3/users/f917f91eeb1644acb46f8a734cdf73e0` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:50.366258 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaec7ed-45e7-452c-b114-4278e822e801 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f917f91eeb1644acb46f8a734cdf73e0)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:50.369210 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaec7ed-45e7-452c-b114-4278e822e801 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:50.425151 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-5eaec7ed-45e7-452c-b114-4278e822e801 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f917f91eeb1644acb46f8a734cdf73e0'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:50.425777 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-5eaec7ed-45e7-452c-b114-4278e822e801 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f917f91eeb1644acb46f8a734cdf73e0'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:50.437949 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-5eaec7ed-45e7-452c-b114-4278e822e801 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f917f91eeb1644acb46f8a734cdf73e0'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:50.446840 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-5eaec7ed-45e7-452c-b114-4278e822e801 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f917f91eeb1644acb46f8a734cdf73e0'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:50.467144 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-5eaec7ed-45e7-452c-b114-4278e822e801 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f917f91eeb1644acb46f8a734cdf73e0'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:50.476160 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1256/2519] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:15:50 2022] DELETE /identity/v3/users/f917f91eeb1644acb46f8a734cdf73e0 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:15:50.682679 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96168f35-a2fa-4398-a120-988e8d763eef None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:50.683111 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96168f35-a2fa-4398-a120-988e8d763eef None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:50.683387 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96168f35-a2fa-4398-a120-988e8d763eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:50.704152 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-96168f35-a2fa-4398-a120-988e8d763eef None None] MFA Rules not processed for user `e07e83c0bdba47d39c09308eba1bdaae`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:50.705459 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-96168f35-a2fa-4398-a120-988e8d763eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:50.766284 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1264/2520] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Aug 1 14:15:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:15:50.780168 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaec7ed-45e7-452c-b114-4278e822e801 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:50.781715 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5eaec7ed-45e7-452c-b114-4278e822e801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:50.792852 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96168f35-a2fa-4398-a120-988e8d763eef None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:50.794664 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-96168f35-a2fa-4398-a120-988e8d763eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:50.876759 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaec7ed-45e7-452c-b114-4278e822e801 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:50.882804 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb152fc3-3c57-464b-8c7d-1cb20db19c1f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cda7f365ba0405daaf86870960f12d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:50.883002 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96168f35-a2fa-4398-a120-988e8d763eef None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:50.883484 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb152fc3-3c57-464b-8c7d-1cb20db19c1f service swift] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:50.883975 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb152fc3-3c57-464b-8c7d-1cb20db19c1f service swift] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:50.884366 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb152fc3-3c57-464b-8c7d-1cb20db19c1f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:50.888250 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df0311-b08a-48f6-ad87-88721d017e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:50.888956 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df0311-b08a-48f6-ad87-88721d017e24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:50.889236 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df0311-b08a-48f6-ad87-88721d017e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:50.889655 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df0311-b08a-48f6-ad87-88721d017e24 admin admin] PATH_INFO: `/v3/projects/b7ebace868cb4f5fafa8531836e754bc` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:50.890739 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df0311-b08a-48f6-ad87-88721d017e24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7ebace868cb4f5fafa8531836e754bc)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:50.892739 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df0311-b08a-48f6-ad87-88721d017e24 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:50.893258 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb152fc3-3c57-464b-8c7d-1cb20db19c1f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:50.894890 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb152fc3-3c57-464b-8c7d-1cb20db19c1f service swift] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:50.904348 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1257/2521] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 14:15:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:15:50.951760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b5df0311-b08a-48f6-ad87-88721d017e24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7ebace868cb4f5fafa8531836e754bc', 'request_id': 'req-b5df0311-b08a-48f6-ad87-88721d017e24'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:50.958350 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b5df0311-b08a-48f6-ad87-88721d017e24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7ebace868cb4f5fafa8531836e754bc', 'request_id': 'req-b5df0311-b08a-48f6-ad87-88721d017e24'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:50.961048 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:50.972454 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b5df0311-b08a-48f6-ad87-88721d017e24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7ebace868cb4f5fafa8531836e754bc', 'request_id': 'req-b5df0311-b08a-48f6-ad87-88721d017e24'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:50.990104 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b5df0311-b08a-48f6-ad87-88721d017e24 admin admin] The token cache is being invalidate because project b7ebace868cb4f5fafa8531836e754bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:15:50.990406 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b5df0311-b08a-48f6-ad87-88721d017e24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:50.991505 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b5df0311-b08a-48f6-ad87-88721d017e24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:50.993419 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1265/2522] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:15:50 2022] DELETE /identity/v3/projects/b7ebace868cb4f5fafa8531836e754bc => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:15:50.998956 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedfb89-b00b-4dde-a18f-9a245c0b5364 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:51.000418 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedfb89-b00b-4dde-a18f-9a245c0b5364 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:51.000418 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedfb89-b00b-4dde-a18f-9a245c0b5364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:51.028801 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-4bedfb89-b00b-4dde-a18f-9a245c0b5364 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:51.030174 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-4bedfb89-b00b-4dde-a18f-9a245c0b5364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:51.158025 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:51.222440 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1258/2523] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:15:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:15:51.230886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0da57-8875-4e82-97d4-5ff818dd2d3a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:51.231752 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0da57-8875-4e82-97d4-5ff818dd2d3a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:51.231752 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0da57-8875-4e82-97d4-5ff818dd2d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:51.254690 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-68d0da57-8875-4e82-97d4-5ff818dd2d3a None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:51.255916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-68d0da57-8875-4e82-97d4-5ff818dd2d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:51.267002 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1266/2524] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:15:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:15:51.272120 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bedfb89-b00b-4dde-a18f-9a245c0b5364 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:51.285561 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bedfb89-b00b-4dde-a18f-9a245c0b5364 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:51.292304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743accd9-e638-4e48-89a9-d9b4e782bfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:51.293121 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743accd9-e638-4e48-89a9-d9b4e782bfeb admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:51.293519 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743accd9-e638-4e48-89a9-d9b4e782bfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:51.294109 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743accd9-e638-4e48-89a9-d9b4e782bfeb admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:51.295082 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743accd9-e638-4e48-89a9-d9b4e782bfeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:15:51.295688 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743accd9-e638-4e48-89a9-d9b4e782bfeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:51.297749 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743accd9-e638-4e48-89a9-d9b4e782bfeb admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:51.318642 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1259/2525] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:15:51.319749 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0da57-8875-4e82-97d4-5ff818dd2d3a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:51.333187 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0da57-8875-4e82-97d4-5ff818dd2d3a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:51.340230 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1b305-4703-4337-b230-2e04c80e95cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:51.341889 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1b305-4703-4337-b230-2e04c80e95cc admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:51.342677 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1b305-4703-4337-b230-2e04c80e95cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:51.343113 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1b305-4703-4337-b230-2e04c80e95cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:51.343952 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa1b305-4703-4337-b230-2e04c80e95cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:51.346028 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa1b305-4703-4337-b230-2e04c80e95cc admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:51.385700 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1267/2526] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:15:51 2022] POST /identity/v3/projects => generated 399 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:15:51.390684 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743accd9-e638-4e48-89a9-d9b4e782bfeb admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:51.403361 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743accd9-e638-4e48-89a9-d9b4e782bfeb admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:51.477375 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019a4df-7307-4065-a80c-7c7ba7015c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:51.478038 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019a4df-7307-4065-a80c-7c7ba7015c23 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:51.478321 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019a4df-7307-4065-a80c-7c7ba7015c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:51.478608 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019a4df-7307-4065-a80c-7c7ba7015c23 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:51.479421 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3019a4df-7307-4065-a80c-7c7ba7015c23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:51.481727 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3019a4df-7307-4065-a80c-7c7ba7015c23 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:51.520582 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1260/2527] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:15:51 2022] POST /identity/v3/users => generated 300 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:15:51.523086 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1b305-4703-4337-b230-2e04c80e95cc admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:51.537068 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1b305-4703-4337-b230-2e04c80e95cc admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:51.546795 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d29e367-0d77-4fdb-931d-11b80639b39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:51.547592 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29e367-0d77-4fdb-931d-11b80639b39a admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:51.548096 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29e367-0d77-4fdb-931d-11b80639b39a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:51.548438 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29e367-0d77-4fdb-931d-11b80639b39a admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:51.549167 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d29e367-0d77-4fdb-931d-11b80639b39a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:51.551161 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d29e367-0d77-4fdb-931d-11b80639b39a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:51.564105 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1268/2528] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:15:51.568176 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019a4df-7307-4065-a80c-7c7ba7015c23 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:51.584963 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019a4df-7307-4065-a80c-7c7ba7015c23 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:51.603284 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a872c844-547d-4923-95b6-8425fe036453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:51.603966 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872c844-547d-4923-95b6-8425fe036453 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:51.604236 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872c844-547d-4923-95b6-8425fe036453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:51.604535 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872c844-547d-4923-95b6-8425fe036453 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:51.607371 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a872c844-547d-4923-95b6-8425fe036453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:51.612426 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a872c844-547d-4923-95b6-8425fe036453 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:51.632027 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1261/2529] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:15:51.636112 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d29e367-0d77-4fdb-931d-11b80639b39a admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:51.648797 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d29e367-0d77-4fdb-931d-11b80639b39a admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:51.655489 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a077eba-976b-42cb-9b82-e09018bc8229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:51.657071 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a077eba-976b-42cb-9b82-e09018bc8229 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:51.657071 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a077eba-976b-42cb-9b82-e09018bc8229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:51.657232 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a077eba-976b-42cb-9b82-e09018bc8229 admin admin] PATH_INFO: `/v3/projects/892587ccada547a7bc7701a852c516e4/users/220ef04ad3ca467ba2721ea9e17ed46e/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:51.680762 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a077eba-976b-42cb-9b82-e09018bc8229 admin admin] RBAC: Authorizing `identity:create_grant(project_id=892587ccada547a7bc7701a852c516e4, user_id=220ef04ad3ca467ba2721ea9e17ed46e, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:51.683547 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a077eba-976b-42cb-9b82-e09018bc8229 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:51.710074 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1269/2530] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:15:51 2022] PUT /identity/v3/projects/892587ccada547a7bc7701a852c516e4/users/220ef04ad3ca467ba2721ea9e17ed46e/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:15:51.724225 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3513b-de24-49c7-a5fd-89ff3649614b None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:51.724679 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3513b-de24-49c7-a5fd-89ff3649614b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:51.725037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3513b-de24-49c7-a5fd-89ff3649614b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:51.771507 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-09b3513b-de24-49c7-a5fd-89ff3649614b None None] MFA Rules not processed for user `220ef04ad3ca467ba2721ea9e17ed46e`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:51.772871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-09b3513b-de24-49c7-a5fd-89ff3649614b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:51.868826 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1262/2531] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:15:51 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:15:51.873574 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a077eba-976b-42cb-9b82-e09018bc8229 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:51.949611 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a077eba-976b-42cb-9b82-e09018bc8229 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:51.955067 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b8245-e570-43a0-8d19-1e43b3f005ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:51.955818 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b8245-e570-43a0-8d19-1e43b3f005ff admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:51.956110 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b8245-e570-43a0-8d19-1e43b3f005ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:51.956405 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b8245-e570-43a0-8d19-1e43b3f005ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:51.957085 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b8245-e570-43a0-8d19-1e43b3f005ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:51.959035 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b8245-e570-43a0-8d19-1e43b3f005ff admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:51.990293 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1270/2532] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:15:51 2022] POST /identity/v3/projects => generated 399 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:15:51.994885 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3513b-de24-49c7-a5fd-89ff3649614b None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:52.007269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3513b-de24-49c7-a5fd-89ff3649614b None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:52.062983 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:52.090551 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ee5a5-eff9-4699-b7fd-e8008f3d9251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:52.091105 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ee5a5-eff9-4699-b7fd-e8008f3d9251 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:52.091587 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ee5a5-eff9-4699-b7fd-e8008f3d9251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:52.091698 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ee5a5-eff9-4699-b7fd-e8008f3d9251 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:52.092534 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07ee5a5-eff9-4699-b7fd-e8008f3d9251 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:52.094486 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07ee5a5-eff9-4699-b7fd-e8008f3d9251 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:52.120724 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1263/2533] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:15:51 2022] POST /identity/v3/users => generated 306 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:15:52.124737 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b8245-e570-43a0-8d19-1e43b3f005ff admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:52.136246 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b8245-e570-43a0-8d19-1e43b3f005ff admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:52.141571 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497eb4e9-40de-4223-8ffb-297850e6b44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:52.142296 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497eb4e9-40de-4223-8ffb-297850e6b44a admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:52.142649 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497eb4e9-40de-4223-8ffb-297850e6b44a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:52.143021 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497eb4e9-40de-4223-8ffb-297850e6b44a admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:52.143786 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497eb4e9-40de-4223-8ffb-297850e6b44a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:52.145534 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497eb4e9-40de-4223-8ffb-297850e6b44a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:52.157692 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1271/2534] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:15:52.162441 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ee5a5-eff9-4699-b7fd-e8008f3d9251 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:52.174873 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ee5a5-eff9-4699-b7fd-e8008f3d9251 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:52.180916 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fbee3-8bc4-49aa-8070-2643e885215a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:52.181658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fbee3-8bc4-49aa-8070-2643e885215a admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:52.181941 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fbee3-8bc4-49aa-8070-2643e885215a admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:52.182224 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fbee3-8bc4-49aa-8070-2643e885215a admin admin] PATH_INFO: `/v3/projects/be0bd3f033a743e18f72008757b6833a/users/cbc7e97dbb1040c18f6998020b386611/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:52.200631 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4fbee3-8bc4-49aa-8070-2643e885215a admin admin] RBAC: Authorizing `identity:create_grant(project_id=be0bd3f033a743e18f72008757b6833a, user_id=cbc7e97dbb1040c18f6998020b386611, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:52.203308 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4fbee3-8bc4-49aa-8070-2643e885215a admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:52.224104 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1264/2535] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:15:52 2022] PUT /identity/v3/projects/be0bd3f033a743e18f72008757b6833a/users/cbc7e97dbb1040c18f6998020b386611/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:15:52.225666 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497eb4e9-40de-4223-8ffb-297850e6b44a admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:52.238556 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497eb4e9-40de-4223-8ffb-297850e6b44a admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:52.304457 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:52.313699 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a93159-2715-490b-83f6-11b0f0420e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:52.314427 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a93159-2715-490b-83f6-11b0f0420e75 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:52.314828 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a93159-2715-490b-83f6-11b0f0420e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:52.314988 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a93159-2715-490b-83f6-11b0f0420e75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:52.315580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a93159-2715-490b-83f6-11b0f0420e75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:52.317230 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a93159-2715-490b-83f6-11b0f0420e75 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:52.331789 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1272/2536] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:15:52.337747 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fbee3-8bc4-49aa-8070-2643e885215a admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:52.350250 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fbee3-8bc4-49aa-8070-2643e885215a admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:52.355626 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d603be5e-ab9a-4e44-af1a-d5446cf36133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:52.356367 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603be5e-ab9a-4e44-af1a-d5446cf36133 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:52.356677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603be5e-ab9a-4e44-af1a-d5446cf36133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:52.356956 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603be5e-ab9a-4e44-af1a-d5446cf36133 admin admin] PATH_INFO: `/v3/projects/be0bd3f033a743e18f72008757b6833a/users/cbc7e97dbb1040c18f6998020b386611/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:52.363491 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d603be5e-ab9a-4e44-af1a-d5446cf36133 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be0bd3f033a743e18f72008757b6833a, user_id=cbc7e97dbb1040c18f6998020b386611, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:52.363491 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d603be5e-ab9a-4e44-af1a-d5446cf36133 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:52.375961 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1265/2537] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:15:52 2022] PUT /identity/v3/projects/be0bd3f033a743e18f72008757b6833a/users/cbc7e97dbb1040c18f6998020b386611/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:15:52.389033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac612cb-7988-485e-8a12-4f2490925e6e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:52.389584 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac612cb-7988-485e-8a12-4f2490925e6e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:52.389924 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac612cb-7988-485e-8a12-4f2490925e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:52.423218 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-6ac612cb-7988-485e-8a12-4f2490925e6e None None] MFA Rules not processed for user `cbc7e97dbb1040c18f6998020b386611`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:52.427612 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6ac612cb-7988-485e-8a12-4f2490925e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:52.508896 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1273/2538] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:15:52 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:15:52.521295 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d603be5e-ab9a-4e44-af1a-d5446cf36133 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:52.524236 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d603be5e-ab9a-4e44-af1a-d5446cf36133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:52.623047 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d603be5e-ab9a-4e44-af1a-d5446cf36133 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:52.625006 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a36295d-1f73-4642-9803-283b77f5bdcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:52.625648 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36295d-1f73-4642-9803-283b77f5bdcf service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:52.625921 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36295d-1f73-4642-9803-283b77f5bdcf service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:52.626274 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36295d-1f73-4642-9803-283b77f5bdcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:52.636558 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a36295d-1f73-4642-9803-283b77f5bdcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:52.638015 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a36295d-1f73-4642-9803-283b77f5bdcf service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:52.652035 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1266/2539] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:15:52 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:15:52.751444 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac612cb-7988-485e-8a12-4f2490925e6e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:52.766688 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac612cb-7988-485e-8a12-4f2490925e6e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:52.777310 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeadded-d2f2-4f83-8189-e5a2d3c5d61d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:52.777310 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeadded-d2f2-4f83-8189-e5a2d3c5d61d service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:52.777310 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeadded-d2f2-4f83-8189-e5a2d3c5d61d service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:52.777310 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeadded-d2f2-4f83-8189-e5a2d3c5d61d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:52.855391 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeadded-d2f2-4f83-8189-e5a2d3c5d61d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:52.857090 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeadded-d2f2-4f83-8189-e5a2d3c5d61d service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:52.868749 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1274/2540] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:15:52 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:15:57.195643 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a36295d-1f73-4642-9803-283b77f5bdcf service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:57.269620 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a36295d-1f73-4642-9803-283b77f5bdcf service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:57.275143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3408f-baf6-4a7f-80a4-79fea9449ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:57.276044 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3408f-baf6-4a7f-80a4-79fea9449ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:57.276428 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3408f-baf6-4a7f-80a4-79fea9449ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:57.276792 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3408f-baf6-4a7f-80a4-79fea9449ff6 admin admin] PATH_INFO: `/v3/users/220ef04ad3ca467ba2721ea9e17ed46e` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:57.278081 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3408f-baf6-4a7f-80a4-79fea9449ff6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=220ef04ad3ca467ba2721ea9e17ed46e)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:57.280147 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3408f-baf6-4a7f-80a4-79fea9449ff6 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:57.330337 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:57.342362 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-21e3408f-baf6-4a7f-80a4-79fea9449ff6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '220ef04ad3ca467ba2721ea9e17ed46e'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:57.343427 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-21e3408f-baf6-4a7f-80a4-79fea9449ff6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '220ef04ad3ca467ba2721ea9e17ed46e'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:57.353701 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-21e3408f-baf6-4a7f-80a4-79fea9449ff6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '220ef04ad3ca467ba2721ea9e17ed46e'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:57.362507 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-21e3408f-baf6-4a7f-80a4-79fea9449ff6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '220ef04ad3ca467ba2721ea9e17ed46e'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:57.380113 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-21e3408f-baf6-4a7f-80a4-79fea9449ff6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '220ef04ad3ca467ba2721ea9e17ed46e'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:57.389197 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1267/2541] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:15:57 2022] DELETE /identity/v3/users/220ef04ad3ca467ba2721ea9e17ed46e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:15:57.394236 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eeb2c7-79c1-4150-99de-37a8a396b2f2 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:57.394829 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eeb2c7-79c1-4150-99de-37a8a396b2f2 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:57.394829 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eeb2c7-79c1-4150-99de-37a8a396b2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:57.420623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a2eeb2c7-79c1-4150-99de-37a8a396b2f2 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:57.421878 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a2eeb2c7-79c1-4150-99de-37a8a396b2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:57.500882 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1275/2542] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:15:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:15:57.508149 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3408f-baf6-4a7f-80a4-79fea9449ff6 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:57.509767 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-21e3408f-baf6-4a7f-80a4-79fea9449ff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:57.621236 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3408f-baf6-4a7f-80a4-79fea9449ff6 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:57.627623 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44265b58-5c18-4baf-a96f-f5ad2da9ad1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:57.628511 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44265b58-5c18-4baf-a96f-f5ad2da9ad1f service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:57.628971 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44265b58-5c18-4baf-a96f-f5ad2da9ad1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:57.629523 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44265b58-5c18-4baf-a96f-f5ad2da9ad1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:57.644519 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44265b58-5c18-4baf-a96f-f5ad2da9ad1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:57.646033 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44265b58-5c18-4baf-a96f-f5ad2da9ad1f service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:57.660147 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1268/2543] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:15:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:15:58.139622 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eeb2c7-79c1-4150-99de-37a8a396b2f2 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:58.142070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a2eeb2c7-79c1-4150-99de-37a8a396b2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:58.167194 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eeb2c7-79c1-4150-99de-37a8a396b2f2 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:58.174783 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe80ac2-12a9-4b47-b611-a6d2b5158d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:58.175658 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe80ac2-12a9-4b47-b611-a6d2b5158d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:58.176113 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe80ac2-12a9-4b47-b611-a6d2b5158d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:58.176418 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe80ac2-12a9-4b47-b611-a6d2b5158d28 admin admin] PATH_INFO: `/v3/projects/892587ccada547a7bc7701a852c516e4` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:58.177684 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe80ac2-12a9-4b47-b611-a6d2b5158d28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=892587ccada547a7bc7701a852c516e4)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:58.179933 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe80ac2-12a9-4b47-b611-a6d2b5158d28 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:58.209630 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:58.281492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6fe80ac2-12a9-4b47-b611-a6d2b5158d28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '892587ccada547a7bc7701a852c516e4', 'request_id': 'req-6fe80ac2-12a9-4b47-b611-a6d2b5158d28'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:58.290936 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6fe80ac2-12a9-4b47-b611-a6d2b5158d28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '892587ccada547a7bc7701a852c516e4', 'request_id': 'req-6fe80ac2-12a9-4b47-b611-a6d2b5158d28'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:58.300950 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6fe80ac2-12a9-4b47-b611-a6d2b5158d28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '892587ccada547a7bc7701a852c516e4', 'request_id': 'req-6fe80ac2-12a9-4b47-b611-a6d2b5158d28'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:58.317351 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6fe80ac2-12a9-4b47-b611-a6d2b5158d28 admin admin] The token cache is being invalidate because project 892587ccada547a7bc7701a852c516e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:15:58.317786 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6fe80ac2-12a9-4b47-b611-a6d2b5158d28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:58.318323 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6fe80ac2-12a9-4b47-b611-a6d2b5158d28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:58.325835 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1276/2544] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:15:58 2022] DELETE /identity/v3/projects/892587ccada547a7bc7701a852c516e4 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:15:58.329796 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44265b58-5c18-4baf-a96f-f5ad2da9ad1f service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:58.353103 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44265b58-5c18-4baf-a96f-f5ad2da9ad1f service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:58.453274 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c6559-5136-4d38-b19d-fd58ac3f2b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:58.454268 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c6559-5136-4d38-b19d-fd58ac3f2b8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:58.454539 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c6559-5136-4d38-b19d-fd58ac3f2b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:58.454809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c6559-5136-4d38-b19d-fd58ac3f2b8d admin admin] PATH_INFO: `/v3/users/cbc7e97dbb1040c18f6998020b386611` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:58.456064 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556c6559-5136-4d38-b19d-fd58ac3f2b8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbc7e97dbb1040c18f6998020b386611)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:58.458106 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556c6559-5136-4d38-b19d-fd58ac3f2b8d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:58.517985 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-556c6559-5136-4d38-b19d-fd58ac3f2b8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbc7e97dbb1040c18f6998020b386611'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:58.518687 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-556c6559-5136-4d38-b19d-fd58ac3f2b8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbc7e97dbb1040c18f6998020b386611'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:58.530249 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-556c6559-5136-4d38-b19d-fd58ac3f2b8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbc7e97dbb1040c18f6998020b386611'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:58.540927 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-556c6559-5136-4d38-b19d-fd58ac3f2b8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbc7e97dbb1040c18f6998020b386611'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:58.554248 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:58.565754 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-556c6559-5136-4d38-b19d-fd58ac3f2b8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbc7e97dbb1040c18f6998020b386611'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:58.575404 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1269/2545] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:15:58 2022] DELETE /identity/v3/users/cbc7e97dbb1040c18f6998020b386611 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:15:58.917693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe80ac2-12a9-4b47-b611-a6d2b5158d28 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:58.919887 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6fe80ac2-12a9-4b47-b611-a6d2b5158d28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:58.936331 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe80ac2-12a9-4b47-b611-a6d2b5158d28 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:59.062853 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe851903-0d1e-4c83-a834-ea1211327efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:59.063801 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe851903-0d1e-4c83-a834-ea1211327efd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:59.064265 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe851903-0d1e-4c83-a834-ea1211327efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:59.064593 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe851903-0d1e-4c83-a834-ea1211327efd admin admin] PATH_INFO: `/v3/projects/be0bd3f033a743e18f72008757b6833a` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:59.066012 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe851903-0d1e-4c83-a834-ea1211327efd admin admin] RBAC: Authorizing `identity:delete_project(project_id=be0bd3f033a743e18f72008757b6833a)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:59.068102 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe851903-0d1e-4c83-a834-ea1211327efd admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:59.178782 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fe851903-0d1e-4c83-a834-ea1211327efd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be0bd3f033a743e18f72008757b6833a', 'request_id': 'req-fe851903-0d1e-4c83-a834-ea1211327efd'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:59.202334 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fe851903-0d1e-4c83-a834-ea1211327efd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be0bd3f033a743e18f72008757b6833a', 'request_id': 'req-fe851903-0d1e-4c83-a834-ea1211327efd'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:59.227695 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fe851903-0d1e-4c83-a834-ea1211327efd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be0bd3f033a743e18f72008757b6833a', 'request_id': 'req-fe851903-0d1e-4c83-a834-ea1211327efd'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:59.241462 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fe851903-0d1e-4c83-a834-ea1211327efd admin admin] The token cache is being invalidate because project be0bd3f033a743e18f72008757b6833a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:15:59.241819 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fe851903-0d1e-4c83-a834-ea1211327efd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:59.242708 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-fe851903-0d1e-4c83-a834-ea1211327efd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:59.244319 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1277/2546] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:15:58 2022] DELETE /identity/v3/projects/be0bd3f033a743e18f72008757b6833a => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:15:59.249572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa906b5-429d-4f65-907a-5a1f7044dc74 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:59.249858 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa906b5-429d-4f65-907a-5a1f7044dc74 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:59.250127 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa906b5-429d-4f65-907a-5a1f7044dc74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:59.273652 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-eaa906b5-429d-4f65-907a-5a1f7044dc74 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:59.275142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-eaa906b5-429d-4f65-907a-5a1f7044dc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:59.364872 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1270/2547] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:15:59.367403 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa30c2-7bc8-4419-8c66-8cb8c5f6ec3f None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:59.367869 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa30c2-7bc8-4419-8c66-8cb8c5f6ec3f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:59.368321 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa30c2-7bc8-4419-8c66-8cb8c5f6ec3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:59.391870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-aafa30c2-7bc8-4419-8c66-8cb8c5f6ec3f None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:59.393456 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-aafa30c2-7bc8-4419-8c66-8cb8c5f6ec3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:59.402260 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1278/2548] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:15:59.406935 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa906b5-429d-4f65-907a-5a1f7044dc74 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:59.419565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa906b5-429d-4f65-907a-5a1f7044dc74 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:59.424551 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de153268-5fdc-4459-922b-ced5bcf8bc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:59.425202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de153268-5fdc-4459-922b-ced5bcf8bc60 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:59.425490 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de153268-5fdc-4459-922b-ced5bcf8bc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:59.425901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de153268-5fdc-4459-922b-ced5bcf8bc60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:59.426636 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de153268-5fdc-4459-922b-ced5bcf8bc60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:15:59.426949 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de153268-5fdc-4459-922b-ced5bcf8bc60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:59.428686 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de153268-5fdc-4459-922b-ced5bcf8bc60 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:59.441866 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1271/2549] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:15:59.446910 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa30c2-7bc8-4419-8c66-8cb8c5f6ec3f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:59.458906 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa30c2-7bc8-4419-8c66-8cb8c5f6ec3f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:59.471237 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c7b594-3f11-46a1-b021-943d49d06332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:59.471237 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7b594-3f11-46a1-b021-943d49d06332 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:59.471237 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7b594-3f11-46a1-b021-943d49d06332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:59.471237 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7b594-3f11-46a1-b021-943d49d06332 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:59.471743 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c7b594-3f11-46a1-b021-943d49d06332 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:59.471743 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c7b594-3f11-46a1-b021-943d49d06332 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:59.518198 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1279/2550] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:15:59 2022] POST /identity/v3/projects => generated 359 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:15:59.519384 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de153268-5fdc-4459-922b-ced5bcf8bc60 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:59.530599 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de153268-5fdc-4459-922b-ced5bcf8bc60 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:59.617066 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2057d-f873-4e91-855c-80392446eeb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:59.617898 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2057d-f873-4e91-855c-80392446eeb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:59.618312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2057d-f873-4e91-855c-80392446eeb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:59.618764 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2057d-f873-4e91-855c-80392446eeb9 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:59.619624 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda2057d-f873-4e91-855c-80392446eeb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:59.621853 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda2057d-f873-4e91-855c-80392446eeb9 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:59.646180 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1272/2551] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:15:59 2022] POST /identity/v3/users => generated 280 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:15:59.650757 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c7b594-3f11-46a1-b021-943d49d06332 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:59.656553 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:59.663597 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c7b594-3f11-46a1-b021-943d49d06332 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:59.670419 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3405fd40-eb55-4446-a321-3de4af431ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:59.671155 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3405fd40-eb55-4446-a321-3de4af431ef9 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:59.671602 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3405fd40-eb55-4446-a321-3de4af431ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:59.672017 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3405fd40-eb55-4446-a321-3de4af431ef9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:59.672983 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3405fd40-eb55-4446-a321-3de4af431ef9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:59.675071 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3405fd40-eb55-4446-a321-3de4af431ef9 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:59.687269 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1280/2552] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:15:59.691687 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2057d-f873-4e91-855c-80392446eeb9 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:59.702433 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2057d-f873-4e91-855c-80392446eeb9 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:59.707240 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0a843-667b-4be0-be49-7fcbf194833b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:59.707919 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0a843-667b-4be0-be49-7fcbf194833b admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:59.708202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0a843-667b-4be0-be49-7fcbf194833b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:59.708511 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0a843-667b-4be0-be49-7fcbf194833b admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:59.709216 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e0a843-667b-4be0-be49-7fcbf194833b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:59.710967 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e0a843-667b-4be0-be49-7fcbf194833b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:59.722398 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1273/2553] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:15:59.726754 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3405fd40-eb55-4446-a321-3de4af431ef9 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:59.737975 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3405fd40-eb55-4446-a321-3de4af431ef9 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:59.743882 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965278c5-b8cc-4b9c-a4eb-ca75d58f7cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:59.744601 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965278c5-b8cc-4b9c-a4eb-ca75d58f7cae admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:59.744882 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965278c5-b8cc-4b9c-a4eb-ca75d58f7cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:59.745138 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965278c5-b8cc-4b9c-a4eb-ca75d58f7cae admin admin] PATH_INFO: `/v3/projects/4866182bc1354255b797e11296e404d5/users/3143b792a874458c8fb4ab357c4e3752/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:59.763027 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965278c5-b8cc-4b9c-a4eb-ca75d58f7cae admin admin] RBAC: Authorizing `identity:create_grant(project_id=4866182bc1354255b797e11296e404d5, user_id=3143b792a874458c8fb4ab357c4e3752, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:59.765731 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965278c5-b8cc-4b9c-a4eb-ca75d58f7cae admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:59.782297 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1281/2554] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:15:59 2022] PUT /identity/v3/projects/4866182bc1354255b797e11296e404d5/users/3143b792a874458c8fb4ab357c4e3752/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:15:59.794159 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21dc60-362c-4872-97d8-3047c06716b9 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:59.794484 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21dc60-362c-4872-97d8-3047c06716b9 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:59.794944 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21dc60-362c-4872-97d8-3047c06716b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:59.827334 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-8e21dc60-362c-4872-97d8-3047c06716b9 None None] MFA Rules not processed for user `3143b792a874458c8fb4ab357c4e3752`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:59.828619 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-8e21dc60-362c-4872-97d8-3047c06716b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:59.865657 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:59.906937 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1274/2555] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:15:59 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:15:59.942760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965278c5-b8cc-4b9c-a4eb-ca75d58f7cae admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:59.947868 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-965278c5-b8cc-4b9c-a4eb-ca75d58f7cae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:00.010240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965278c5-b8cc-4b9c-a4eb-ca75d58f7cae admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:00.014047 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e5804e-7ef4-4a49-8925-22645341b3de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:00.014710 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5804e-7ef4-4a49-8925-22645341b3de service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:00.015111 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5804e-7ef4-4a49-8925-22645341b3de service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:00.015306 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5804e-7ef4-4a49-8925-22645341b3de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:00.026789 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e5804e-7ef4-4a49-8925-22645341b3de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:00.028247 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e5804e-7ef4-4a49-8925-22645341b3de service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:00.038164 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1282/2556] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:15:59 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:16:00.054695 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21dc60-362c-4872-97d8-3047c06716b9 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:00.122346 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21dc60-362c-4872-97d8-3047c06716b9 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:00.127282 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb437bb6-1dfb-4f70-93d9-b2cfa09d3f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:00.127964 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb437bb6-1dfb-4f70-93d9-b2cfa09d3f11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:00.128251 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb437bb6-1dfb-4f70-93d9-b2cfa09d3f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:00.128627 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb437bb6-1dfb-4f70-93d9-b2cfa09d3f11 admin admin] PATH_INFO: `/v3/users/3143b792a874458c8fb4ab357c4e3752` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:00.131097 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb437bb6-1dfb-4f70-93d9-b2cfa09d3f11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3143b792a874458c8fb4ab357c4e3752)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:00.134433 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb437bb6-1dfb-4f70-93d9-b2cfa09d3f11 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:00.215837 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-eb437bb6-1dfb-4f70-93d9-b2cfa09d3f11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3143b792a874458c8fb4ab357c4e3752'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:00.216745 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-eb437bb6-1dfb-4f70-93d9-b2cfa09d3f11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3143b792a874458c8fb4ab357c4e3752'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:00.228713 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-eb437bb6-1dfb-4f70-93d9-b2cfa09d3f11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3143b792a874458c8fb4ab357c4e3752'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:00.243608 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-eb437bb6-1dfb-4f70-93d9-b2cfa09d3f11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3143b792a874458c8fb4ab357c4e3752'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:00.262020 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-eb437bb6-1dfb-4f70-93d9-b2cfa09d3f11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3143b792a874458c8fb4ab357c4e3752'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:00.272527 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1275/2557] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:16:00 2022] DELETE /identity/v3/users/3143b792a874458c8fb4ab357c4e3752 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:16:00.276205 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b278d5-6e22-43e4-8101-3630f2a103c2 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:00.276532 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b278d5-6e22-43e4-8101-3630f2a103c2 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:00.276787 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b278d5-6e22-43e4-8101-3630f2a103c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:00.313807 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-67b278d5-6e22-43e4-8101-3630f2a103c2 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:16:00.318579 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-67b278d5-6e22-43e4-8101-3630f2a103c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:00.403015 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1283/2558] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:16:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:16:00.413348 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb437bb6-1dfb-4f70-93d9-b2cfa09d3f11 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:00.415318 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-eb437bb6-1dfb-4f70-93d9-b2cfa09d3f11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:00.487268 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb437bb6-1dfb-4f70-93d9-b2cfa09d3f11 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:00.492736 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aceb16c-ea9f-431b-bd07-2f79f9acbab8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:00.493475 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aceb16c-ea9f-431b-bd07-2f79f9acbab8 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:00.493825 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aceb16c-ea9f-431b-bd07-2f79f9acbab8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:00.494177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aceb16c-ea9f-431b-bd07-2f79f9acbab8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:00.500831 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:16:00.509588 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aceb16c-ea9f-431b-bd07-2f79f9acbab8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:00.511122 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aceb16c-ea9f-431b-bd07-2f79f9acbab8 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:00.522491 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1276/2559] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:16:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:16:00.840137 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b278d5-6e22-43e4-8101-3630f2a103c2 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:00.841686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-67b278d5-6e22-43e4-8101-3630f2a103c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:00.856311 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b278d5-6e22-43e4-8101-3630f2a103c2 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:00.861208 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c1c3c3-fa55-48c3-81ce-c3a5179b6853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:00.862679 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1c3c3-fa55-48c3-81ce-c3a5179b6853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:00.862965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1c3c3-fa55-48c3-81ce-c3a5179b6853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:00.863237 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1c3c3-fa55-48c3-81ce-c3a5179b6853 admin admin] PATH_INFO: `/v3/projects/4866182bc1354255b797e11296e404d5` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:00.864649 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c1c3c3-fa55-48c3-81ce-c3a5179b6853 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4866182bc1354255b797e11296e404d5)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:00.866747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c1c3c3-fa55-48c3-81ce-c3a5179b6853 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:00.918337 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-01c1c3c3-fa55-48c3-81ce-c3a5179b6853 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4866182bc1354255b797e11296e404d5', 'request_id': 'req-01c1c3c3-fa55-48c3-81ce-c3a5179b6853'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:00.924333 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-01c1c3c3-fa55-48c3-81ce-c3a5179b6853 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4866182bc1354255b797e11296e404d5', 'request_id': 'req-01c1c3c3-fa55-48c3-81ce-c3a5179b6853'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:00.933036 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-01c1c3c3-fa55-48c3-81ce-c3a5179b6853 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4866182bc1354255b797e11296e404d5', 'request_id': 'req-01c1c3c3-fa55-48c3-81ce-c3a5179b6853'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:00.934294 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:16:00.948777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-01c1c3c3-fa55-48c3-81ce-c3a5179b6853 admin admin] The token cache is being invalidate because project 4866182bc1354255b797e11296e404d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:16:00.949472 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-01c1c3c3-fa55-48c3-81ce-c3a5179b6853 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:00.949790 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-01c1c3c3-fa55-48c3-81ce-c3a5179b6853 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:00.951361 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1284/2560] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:16:00 2022] DELETE /identity/v3/projects/4866182bc1354255b797e11296e404d5 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:16:00.956498 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96fe948-18fe-4620-8940-52d1e8041fdf None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:00.956892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96fe948-18fe-4620-8940-52d1e8041fdf None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:00.957261 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96fe948-18fe-4620-8940-52d1e8041fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:00.984200 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-a96fe948-18fe-4620-8940-52d1e8041fdf None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:16:00.985500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a96fe948-18fe-4620-8940-52d1e8041fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:01.056145 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1277/2561] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:16:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:16:01.127949 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201b8d6-526e-470f-862d-78ebd0b89ab1 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:01.128412 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201b8d6-526e-470f-862d-78ebd0b89ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:01.129308 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201b8d6-526e-470f-862d-78ebd0b89ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:01.158354 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-9201b8d6-526e-470f-862d-78ebd0b89ab1 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:16:01.159774 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9201b8d6-526e-470f-862d-78ebd0b89ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:01.170504 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1285/2562] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:16:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:16:01.176542 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96fe948-18fe-4620-8940-52d1e8041fdf None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:01.186412 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96fe948-18fe-4620-8940-52d1e8041fdf None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:01.191472 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2601264-fcaf-4321-b34f-eb971b069679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:01.192113 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2601264-fcaf-4321-b34f-eb971b069679 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:01.192425 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2601264-fcaf-4321-b34f-eb971b069679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:01.192697 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2601264-fcaf-4321-b34f-eb971b069679 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:01.193411 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2601264-fcaf-4321-b34f-eb971b069679 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:16:01.193722 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2601264-fcaf-4321-b34f-eb971b069679 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:01.195402 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2601264-fcaf-4321-b34f-eb971b069679 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:01.215544 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1278/2563] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:16:01.221532 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201b8d6-526e-470f-862d-78ebd0b89ab1 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:01.236808 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201b8d6-526e-470f-862d-78ebd0b89ab1 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:01.242146 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c647d32-6d74-4647-bdd2-b01d3b9670ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:01.242785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c647d32-6d74-4647-bdd2-b01d3b9670ea admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:01.243059 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c647d32-6d74-4647-bdd2-b01d3b9670ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:01.243339 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c647d32-6d74-4647-bdd2-b01d3b9670ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:01.243985 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c647d32-6d74-4647-bdd2-b01d3b9670ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:01.245943 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c647d32-6d74-4647-bdd2-b01d3b9670ea admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:01.282845 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1286/2564] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:16:01 2022] POST /identity/v3/projects => generated 377 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:16:01.288470 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2601264-fcaf-4321-b34f-eb971b069679 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:01.301337 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2601264-fcaf-4321-b34f-eb971b069679 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:01.385648 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11cc97-0525-4dbc-b452-79cbbc67714d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:01.386496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11cc97-0525-4dbc-b452-79cbbc67714d admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:01.386496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11cc97-0525-4dbc-b452-79cbbc67714d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:01.386700 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11cc97-0525-4dbc-b452-79cbbc67714d admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:01.387375 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11cc97-0525-4dbc-b452-79cbbc67714d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:01.391137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11cc97-0525-4dbc-b452-79cbbc67714d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:01.423917 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1279/2565] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:16:01 2022] POST /identity/v3/users => generated 289 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:16:01.429250 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c647d32-6d74-4647-bdd2-b01d3b9670ea admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:01.446582 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c647d32-6d74-4647-bdd2-b01d3b9670ea admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:01.452114 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f6e5a-d36f-48da-89db-8f470486d113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:01.453178 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f6e5a-d36f-48da-89db-8f470486d113 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:01.453879 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f6e5a-d36f-48da-89db-8f470486d113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:01.454090 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f6e5a-d36f-48da-89db-8f470486d113 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:01.455172 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f6e5a-d36f-48da-89db-8f470486d113 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:01.456939 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f6e5a-d36f-48da-89db-8f470486d113 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:01.472374 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1287/2566] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:16:01.479541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11cc97-0525-4dbc-b452-79cbbc67714d admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:01.498082 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11cc97-0525-4dbc-b452-79cbbc67714d admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:01.504671 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bfe0e2-ca35-45b1-8e26-67028a830d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:01.505321 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfe0e2-ca35-45b1-8e26-67028a830d29 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:01.505787 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfe0e2-ca35-45b1-8e26-67028a830d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:01.506130 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfe0e2-ca35-45b1-8e26-67028a830d29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:01.506998 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bfe0e2-ca35-45b1-8e26-67028a830d29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:01.508739 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bfe0e2-ca35-45b1-8e26-67028a830d29 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:01.525290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1280/2567] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:16:01.526821 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f6e5a-d36f-48da-89db-8f470486d113 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:01.545764 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f6e5a-d36f-48da-89db-8f470486d113 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:01.550902 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00059dc-e021-4e6c-8f95-f3137bdaf630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:01.551589 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00059dc-e021-4e6c-8f95-f3137bdaf630 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:01.551868 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00059dc-e021-4e6c-8f95-f3137bdaf630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:01.552146 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00059dc-e021-4e6c-8f95-f3137bdaf630 admin admin] PATH_INFO: `/v3/projects/92b3ea16a11a43c19e7c6fccc485e723/users/d388dba404c244b69e944b711a5ef14a/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:01.574557 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00059dc-e021-4e6c-8f95-f3137bdaf630 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92b3ea16a11a43c19e7c6fccc485e723, user_id=d388dba404c244b69e944b711a5ef14a, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:01.577662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00059dc-e021-4e6c-8f95-f3137bdaf630 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:01.600325 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1288/2568] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:16:01 2022] PUT /identity/v3/projects/92b3ea16a11a43c19e7c6fccc485e723/users/d388dba404c244b69e944b711a5ef14a/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:16:01.607652 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d14947-e66a-440e-9bd6-b95acc1ac52b None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:01.607940 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d14947-e66a-440e-9bd6-b95acc1ac52b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:01.608214 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d14947-e66a-440e-9bd6-b95acc1ac52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:01.648630 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-34d14947-e66a-440e-9bd6-b95acc1ac52b None None] MFA Rules not processed for user `d388dba404c244b69e944b711a5ef14a`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:16:01.648630 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-34d14947-e66a-440e-9bd6-b95acc1ac52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:01.717402 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:16:01.738991 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1281/2569] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:16:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:16:01.751705 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00059dc-e021-4e6c-8f95-f3137bdaf630 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:01.753645 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d00059dc-e021-4e6c-8f95-f3137bdaf630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:01.817352 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00059dc-e021-4e6c-8f95-f3137bdaf630 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:01.823558 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec84f3-1124-4c6a-a273-bb9070e41cb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:01.824252 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec84f3-1124-4c6a-a273-bb9070e41cb2 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:01.824557 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec84f3-1124-4c6a-a273-bb9070e41cb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:01.824859 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec84f3-1124-4c6a-a273-bb9070e41cb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:01.836108 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ec84f3-1124-4c6a-a273-bb9070e41cb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:01.838824 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ec84f3-1124-4c6a-a273-bb9070e41cb2 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:01.852984 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1289/2570] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:16:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:16:11.491303 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae452c-4309-4fb0-9577-d13a6c2b5f79 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:11.491807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae452c-4309-4fb0-9577-d13a6c2b5f79 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:11.491900 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae452c-4309-4fb0-9577-d13a6c2b5f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:11.516646 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-73ae452c-4309-4fb0-9577-d13a6c2b5f79 None None] MFA Rules not processed for user `e07e83c0bdba47d39c09308eba1bdaae`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:16:11.517851 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-73ae452c-4309-4fb0-9577-d13a6c2b5f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:11.602152 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1282/2571] 10.222.0.45 () {58 vars in 967 bytes} [Mon Aug 1 14:16:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:16:11.615009 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec84f3-1124-4c6a-a273-bb9070e41cb2 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:11.616568 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-60ec84f3-1124-4c6a-a273-bb9070e41cb2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:11.698496 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec84f3-1124-4c6a-a273-bb9070e41cb2 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:11.703931 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d301fe-d56c-46d9-b92e-6fb3712e649c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cda7f365ba0405daaf86870960f12d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:11.704518 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d301fe-d56c-46d9-b92e-6fb3712e649c service swift] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:11.704800 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d301fe-d56c-46d9-b92e-6fb3712e649c service swift] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:11.705073 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d301fe-d56c-46d9-b92e-6fb3712e649c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:11.715639 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d301fe-d56c-46d9-b92e-6fb3712e649c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:11.717716 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d301fe-d56c-46d9-b92e-6fb3712e649c service swift] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:11.731800 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1290/2572] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 14:16:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:16:21.593717 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=30 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:16:21.725035 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=88,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:16:22.050226 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae452c-4309-4fb0-9577-d13a6c2b5f79 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:22.220559 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae452c-4309-4fb0-9577-d13a6c2b5f79 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:22.226718 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4653cb19-cbd6-4b74-ae10-bde8dd45e301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:22.227420 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653cb19-cbd6-4b74-ae10-bde8dd45e301 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:22.227998 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653cb19-cbd6-4b74-ae10-bde8dd45e301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:22.228337 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653cb19-cbd6-4b74-ae10-bde8dd45e301 admin admin] PATH_INFO: `/v3/users/d388dba404c244b69e944b711a5ef14a` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:22.230130 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4653cb19-cbd6-4b74-ae10-bde8dd45e301 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d388dba404c244b69e944b711a5ef14a)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:22.231891 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4653cb19-cbd6-4b74-ae10-bde8dd45e301 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:22.326274 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4653cb19-cbd6-4b74-ae10-bde8dd45e301 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd388dba404c244b69e944b711a5ef14a'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:22.327229 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4653cb19-cbd6-4b74-ae10-bde8dd45e301 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd388dba404c244b69e944b711a5ef14a'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:22.339469 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4653cb19-cbd6-4b74-ae10-bde8dd45e301 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd388dba404c244b69e944b711a5ef14a'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:22.350104 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4653cb19-cbd6-4b74-ae10-bde8dd45e301 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd388dba404c244b69e944b711a5ef14a'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:22.376686 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-4653cb19-cbd6-4b74-ae10-bde8dd45e301 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd388dba404c244b69e944b711a5ef14a'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:22.378456 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1283/2573] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:16:22 2022] DELETE /identity/v3/users/d388dba404c244b69e944b711a5ef14a => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:16:22.384717 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee54509-94d8-4a1c-b5ff-8739ca1a728b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:22.385134 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee54509-94d8-4a1c-b5ff-8739ca1a728b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:22.385568 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee54509-94d8-4a1c-b5ff-8739ca1a728b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:22.471959 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-0ee54509-94d8-4a1c-b5ff-8739ca1a728b None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:16:22.474500 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0ee54509-94d8-4a1c-b5ff-8739ca1a728b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:22.581334 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1291/2574] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:16:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:16:22.595420 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4653cb19-cbd6-4b74-ae10-bde8dd45e301 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:22.597874 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-4653cb19-cbd6-4b74-ae10-bde8dd45e301 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:22.679498 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:16:22.743590 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4653cb19-cbd6-4b74-ae10-bde8dd45e301 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:22.749794 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1867dd-27dd-4ad6-bbdd-70e4fc874c08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:22.751222 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1867dd-27dd-4ad6-bbdd-70e4fc874c08 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:22.751222 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1867dd-27dd-4ad6-bbdd-70e4fc874c08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:22.751708 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1867dd-27dd-4ad6-bbdd-70e4fc874c08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:22.769063 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1867dd-27dd-4ad6-bbdd-70e4fc874c08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:22.771730 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1867dd-27dd-4ad6-bbdd-70e4fc874c08 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:22.787190 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1284/2575] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:16:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:16:22.900513 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88d898-35cc-49c6-be46-85f0d3e4220a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:22.901353 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88d898-35cc-49c6-be46-85f0d3e4220a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:22.901353 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88d898-35cc-49c6-be46-85f0d3e4220a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:22.927419 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-5f88d898-35cc-49c6-be46-85f0d3e4220a None None] MFA Rules not processed for user `a9fc33bdca304c33b6a78122a54f492a`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:16:22.929210 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5f88d898-35cc-49c6-be46-85f0d3e4220a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:23.017482 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1292/2576] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:16:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:16:23.221646 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1867dd-27dd-4ad6-bbdd-70e4fc874c08 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:23.226127 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-fb1867dd-27dd-4ad6-bbdd-70e4fc874c08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:23.264559 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1867dd-27dd-4ad6-bbdd-70e4fc874c08 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:23.269875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa4a816-ce73-4898-9562-a3e01f54da44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:23.270667 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4a816-ce73-4898-9562-a3e01f54da44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:23.271030 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4a816-ce73-4898-9562-a3e01f54da44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:23.271373 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4a816-ce73-4898-9562-a3e01f54da44 admin admin] PATH_INFO: `/v3/projects/92b3ea16a11a43c19e7c6fccc485e723` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:23.272821 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa4a816-ce73-4898-9562-a3e01f54da44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92b3ea16a11a43c19e7c6fccc485e723)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:23.275160 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa4a816-ce73-4898-9562-a3e01f54da44 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:23.350071 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-faa4a816-ce73-4898-9562-a3e01f54da44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92b3ea16a11a43c19e7c6fccc485e723', 'request_id': 'req-faa4a816-ce73-4898-9562-a3e01f54da44'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:23.358560 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-faa4a816-ce73-4898-9562-a3e01f54da44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92b3ea16a11a43c19e7c6fccc485e723', 'request_id': 'req-faa4a816-ce73-4898-9562-a3e01f54da44'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:23.372708 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-faa4a816-ce73-4898-9562-a3e01f54da44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92b3ea16a11a43c19e7c6fccc485e723', 'request_id': 'req-faa4a816-ce73-4898-9562-a3e01f54da44'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:23.391578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-faa4a816-ce73-4898-9562-a3e01f54da44 admin admin] The token cache is being invalidate because project 92b3ea16a11a43c19e7c6fccc485e723 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:16:23.391891 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-faa4a816-ce73-4898-9562-a3e01f54da44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:23.392747 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-faa4a816-ce73-4898-9562-a3e01f54da44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:23.394505 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1285/2577] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:16:23 2022] DELETE /identity/v3/projects/92b3ea16a11a43c19e7c6fccc485e723 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:16:23.401034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e1505-ef3b-4069-9698-b8f11f07aaf0 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:23.401468 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e1505-ef3b-4069-9698-b8f11f07aaf0 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:23.401866 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e1505-ef3b-4069-9698-b8f11f07aaf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:23.469807 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-004e1505-ef3b-4069-9698-b8f11f07aaf0 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:16:23.470715 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-004e1505-ef3b-4069-9698-b8f11f07aaf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:23.559309 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:16:23.585066 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1293/2578] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:16:23.589643 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7847d-01d7-4c69-8649-d9caf15b3078 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:23.589919 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7847d-01d7-4c69-8649-d9caf15b3078 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:23.590184 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7847d-01d7-4c69-8649-d9caf15b3078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:23.623871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-1bc7847d-01d7-4c69-8649-d9caf15b3078 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:16:23.625179 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1bc7847d-01d7-4c69-8649-d9caf15b3078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:23.633947 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1286/2579] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:16:23.638609 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e1505-ef3b-4069-9698-b8f11f07aaf0 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:23.650001 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e1505-ef3b-4069-9698-b8f11f07aaf0 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:23.656133 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a1f535-9096-4929-86fb-ee30388b00fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:23.657117 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1f535-9096-4929-86fb-ee30388b00fc admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:23.657519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1f535-9096-4929-86fb-ee30388b00fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:23.657801 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1f535-9096-4929-86fb-ee30388b00fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:23.658781 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a1f535-9096-4929-86fb-ee30388b00fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:16:23.659233 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a1f535-9096-4929-86fb-ee30388b00fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:23.661368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a1f535-9096-4929-86fb-ee30388b00fc admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:23.675690 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1294/2580] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:16:23.680487 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc7847d-01d7-4c69-8649-d9caf15b3078 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:23.691368 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc7847d-01d7-4c69-8649-d9caf15b3078 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:23.696862 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d1d92-5861-4b7e-b631-b8c4bd889aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:23.697521 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d1d92-5861-4b7e-b631-b8c4bd889aef admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:23.698381 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d1d92-5861-4b7e-b631-b8c4bd889aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:23.698381 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d1d92-5861-4b7e-b631-b8c4bd889aef admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:23.698744 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856d1d92-5861-4b7e-b631-b8c4bd889aef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:23.700742 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856d1d92-5861-4b7e-b631-b8c4bd889aef admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:23.745213 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1287/2581] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:16:23 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:16:23.747033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a1f535-9096-4929-86fb-ee30388b00fc admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:23.765942 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a1f535-9096-4929-86fb-ee30388b00fc admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:23.849831 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2041cab-ff19-48da-8e96-3e6558803ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:23.850562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2041cab-ff19-48da-8e96-3e6558803ba3 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:23.850779 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2041cab-ff19-48da-8e96-3e6558803ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:23.851013 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2041cab-ff19-48da-8e96-3e6558803ba3 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:23.852070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2041cab-ff19-48da-8e96-3e6558803ba3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:23.854084 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2041cab-ff19-48da-8e96-3e6558803ba3 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:23.891327 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1295/2582] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:16:23 2022] POST /identity/v3/users => generated 288 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:16:23.896338 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d1d92-5861-4b7e-b631-b8c4bd889aef admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:23.912763 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d1d92-5861-4b7e-b631-b8c4bd889aef admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:23.917731 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73816deb-3c6b-4a55-91a7-a6b555056c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:23.918380 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73816deb-3c6b-4a55-91a7-a6b555056c06 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:23.918661 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73816deb-3c6b-4a55-91a7-a6b555056c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:23.918934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73816deb-3c6b-4a55-91a7-a6b555056c06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:23.919700 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73816deb-3c6b-4a55-91a7-a6b555056c06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:23.921426 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73816deb-3c6b-4a55-91a7-a6b555056c06 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:23.933018 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1288/2583] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:16:23.937434 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2041cab-ff19-48da-8e96-3e6558803ba3 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:23.953625 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2041cab-ff19-48da-8e96-3e6558803ba3 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:23.958785 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a59abea-f6ee-4923-8ccb-126b45ef2527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:23.959435 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59abea-f6ee-4923-8ccb-126b45ef2527 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:23.959711 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59abea-f6ee-4923-8ccb-126b45ef2527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:23.960004 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59abea-f6ee-4923-8ccb-126b45ef2527 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:23.960756 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a59abea-f6ee-4923-8ccb-126b45ef2527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:23.962517 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a59abea-f6ee-4923-8ccb-126b45ef2527 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:23.977508 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1296/2584] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:16:23.982271 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73816deb-3c6b-4a55-91a7-a6b555056c06 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:23.995854 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73816deb-3c6b-4a55-91a7-a6b555056c06 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:24.003647 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35290457-e263-44df-a814-fa8d4a88cf54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:24.004493 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35290457-e263-44df-a814-fa8d4a88cf54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:24.005122 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35290457-e263-44df-a814-fa8d4a88cf54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:24.005477 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35290457-e263-44df-a814-fa8d4a88cf54 admin admin] PATH_INFO: `/v3/projects/1049541d859043fe9a31ac86dd01974a/users/c298f2a4128145b38179b9b5c5743fd3/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:24.030929 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35290457-e263-44df-a814-fa8d4a88cf54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1049541d859043fe9a31ac86dd01974a, user_id=c298f2a4128145b38179b9b5c5743fd3, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:24.036315 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35290457-e263-44df-a814-fa8d4a88cf54 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:24.059830 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1289/2585] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:16:23 2022] PUT /identity/v3/projects/1049541d859043fe9a31ac86dd01974a/users/c298f2a4128145b38179b9b5c5743fd3/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:16:24.071999 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fba1fc-ee05-4015-aabf-18601e6cc55e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:24.072371 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fba1fc-ee05-4015-aabf-18601e6cc55e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:24.072904 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fba1fc-ee05-4015-aabf-18601e6cc55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:24.126031 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-e4fba1fc-ee05-4015-aabf-18601e6cc55e None None] MFA Rules not processed for user `c298f2a4128145b38179b9b5c5743fd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:16:24.130039 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e4fba1fc-ee05-4015-aabf-18601e6cc55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:24.248157 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1297/2586] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:16:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:16:24.259495 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35290457-e263-44df-a814-fa8d4a88cf54 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:24.264700 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-35290457-e263-44df-a814-fa8d4a88cf54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:24.361733 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:16:24.383712 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35290457-e263-44df-a814-fa8d4a88cf54 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:24.391244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9713b03-6c55-4f53-97ce-d18e1f00065a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:24.392572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9713b03-6c55-4f53-97ce-d18e1f00065a service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:24.392831 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9713b03-6c55-4f53-97ce-d18e1f00065a service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:24.393065 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9713b03-6c55-4f53-97ce-d18e1f00065a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:24.405342 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9713b03-6c55-4f53-97ce-d18e1f00065a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:24.407198 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9713b03-6c55-4f53-97ce-d18e1f00065a service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:24.420815 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1290/2587] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:16:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:16:34.238062 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=74,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:16:34.414645 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=6 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:16:35.464966 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fba1fc-ee05-4015-aabf-18601e6cc55e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:35.589334 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fba1fc-ee05-4015-aabf-18601e6cc55e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:35.601037 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea7d31-5f2b-4646-a991-8301847383f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:35.602219 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea7d31-5f2b-4646-a991-8301847383f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:35.602730 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea7d31-5f2b-4646-a991-8301847383f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:35.604139 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea7d31-5f2b-4646-a991-8301847383f1 admin admin] PATH_INFO: `/v3/users/c298f2a4128145b38179b9b5c5743fd3` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:35.606797 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eea7d31-5f2b-4646-a991-8301847383f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c298f2a4128145b38179b9b5c5743fd3)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:35.610433 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eea7d31-5f2b-4646-a991-8301847383f1 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:35.697032 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5eea7d31-5f2b-4646-a991-8301847383f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c298f2a4128145b38179b9b5c5743fd3'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:35.697032 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5eea7d31-5f2b-4646-a991-8301847383f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c298f2a4128145b38179b9b5c5743fd3'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:35.711753 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5eea7d31-5f2b-4646-a991-8301847383f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c298f2a4128145b38179b9b5c5743fd3'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:35.723466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5eea7d31-5f2b-4646-a991-8301847383f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c298f2a4128145b38179b9b5c5743fd3'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:35.724240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5eea7d31-5f2b-4646-a991-8301847383f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c298f2a4128145b38179b9b5c5743fd3'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:35.745355 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1298/2588] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:16:35 2022] DELETE /identity/v3/users/c298f2a4128145b38179b9b5c5743fd3 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:16:35.747219 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833ba01-0c4c-432a-8fe7-02097bbc0cdc None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:35.748016 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833ba01-0c4c-432a-8fe7-02097bbc0cdc None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:35.748016 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833ba01-0c4c-432a-8fe7-02097bbc0cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:35.774408 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-f833ba01-0c4c-432a-8fe7-02097bbc0cdc None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:16:35.775714 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-f833ba01-0c4c-432a-8fe7-02097bbc0cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:35.876614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1291/2589] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:16:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:16:35.886196 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea7d31-5f2b-4646-a991-8301847383f1 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:35.887729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5eea7d31-5f2b-4646-a991-8301847383f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:36.021204 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea7d31-5f2b-4646-a991-8301847383f1 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:36.028604 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13daa4-7d70-48e1-9324-b5c2de280566 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:36.029559 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13daa4-7d70-48e1-9324-b5c2de280566 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:36.029840 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13daa4-7d70-48e1-9324-b5c2de280566 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:36.030116 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13daa4-7d70-48e1-9324-b5c2de280566 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:36.044666 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb13daa4-7d70-48e1-9324-b5c2de280566 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:36.047737 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb13daa4-7d70-48e1-9324-b5c2de280566 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:36.059277 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1299/2590] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:16:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:16:36.428199 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f833ba01-0c4c-432a-8fe7-02097bbc0cdc None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:36.430636 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-f833ba01-0c4c-432a-8fe7-02097bbc0cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:36.456114 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f833ba01-0c4c-432a-8fe7-02097bbc0cdc None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:36.463335 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e836f-8254-4ba8-835d-86dccc612afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:36.464202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e836f-8254-4ba8-835d-86dccc612afa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:36.464667 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e836f-8254-4ba8-835d-86dccc612afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:36.465038 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e836f-8254-4ba8-835d-86dccc612afa admin admin] PATH_INFO: `/v3/projects/1049541d859043fe9a31ac86dd01974a` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:36.466561 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e836f-8254-4ba8-835d-86dccc612afa admin admin] RBAC: Authorizing `identity:delete_project(project_id=1049541d859043fe9a31ac86dd01974a)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:36.468785 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e836f-8254-4ba8-835d-86dccc612afa admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:36.559951 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e15e836f-8254-4ba8-835d-86dccc612afa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1049541d859043fe9a31ac86dd01974a', 'request_id': 'req-e15e836f-8254-4ba8-835d-86dccc612afa'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:36.570236 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e15e836f-8254-4ba8-835d-86dccc612afa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1049541d859043fe9a31ac86dd01974a', 'request_id': 'req-e15e836f-8254-4ba8-835d-86dccc612afa'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:36.585731 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e15e836f-8254-4ba8-835d-86dccc612afa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1049541d859043fe9a31ac86dd01974a', 'request_id': 'req-e15e836f-8254-4ba8-835d-86dccc612afa'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:36.603969 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e15e836f-8254-4ba8-835d-86dccc612afa admin admin] The token cache is being invalidate because project 1049541d859043fe9a31ac86dd01974a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:16:36.604397 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e15e836f-8254-4ba8-835d-86dccc612afa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:36.605097 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e15e836f-8254-4ba8-835d-86dccc612afa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:36.607238 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1292/2591] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:16:36 2022] DELETE /identity/v3/projects/1049541d859043fe9a31ac86dd01974a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:16:46.054341 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:16:46.600198 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:16:49.708221 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13daa4-7d70-48e1-9324-b5c2de280566 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:49.710375 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fb13daa4-7d70-48e1-9324-b5c2de280566 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:49.838170 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13daa4-7d70-48e1-9324-b5c2de280566 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:49.909531 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a93c2e-16ea-4edc-95f0-9968919dd07a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58380e30789041e4ad9e8652bc628658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:49.910168 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a93c2e-16ea-4edc-95f0-9968919dd07a service placement] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:49.910447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a93c2e-16ea-4edc-95f0-9968919dd07a service placement] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:49.910713 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a93c2e-16ea-4edc-95f0-9968919dd07a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:49.913106 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f2a93c2e-16ea-4edc-95f0-9968919dd07a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:49.926054 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a93c2e-16ea-4edc-95f0-9968919dd07a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:49.927693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a93c2e-16ea-4edc-95f0-9968919dd07a service placement] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:49.939288 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1300/2592] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:16:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:16:54.068094 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e836f-8254-4ba8-835d-86dccc612afa admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:54.069654 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e15e836f-8254-4ba8-835d-86dccc612afa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:54.145873 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e836f-8254-4ba8-835d-86dccc612afa admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:54.150028 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29922437-2c45-4e63-87a6-73b34556fed0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:54.150681 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29922437-2c45-4e63-87a6-73b34556fed0 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:54.150962 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29922437-2c45-4e63-87a6-73b34556fed0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:54.151231 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29922437-2c45-4e63-87a6-73b34556fed0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:54.153924 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-29922437-2c45-4e63-87a6-73b34556fed0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:54.224260 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29922437-2c45-4e63-87a6-73b34556fed0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:54.225793 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29922437-2c45-4e63-87a6-73b34556fed0 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:54.236613 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1293/2593] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:16:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:16:57.708399 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a93c2e-16ea-4edc-95f0-9968919dd07a service placement] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:57.709904 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f2a93c2e-16ea-4edc-95f0-9968919dd07a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:57.724165 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a93c2e-16ea-4edc-95f0-9968919dd07a service placement] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:57.730565 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6452d7-3e0c-40a7-9fc4-646beb4ef58f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58380e30789041e4ad9e8652bc628658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:57.731715 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6452d7-3e0c-40a7-9fc4-646beb4ef58f service placement] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:57.731715 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6452d7-3e0c-40a7-9fc4-646beb4ef58f service placement] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:57.731715 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6452d7-3e0c-40a7-9fc4-646beb4ef58f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:57.733577 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-be6452d7-3e0c-40a7-9fc4-646beb4ef58f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:57.748863 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6452d7-3e0c-40a7-9fc4-646beb4ef58f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:57.750500 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6452d7-3e0c-40a7-9fc4-646beb4ef58f service placement] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:57.762222 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1301/2594] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:16:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:16:57.775723 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29922437-2c45-4e63-87a6-73b34556fed0 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:57.777476 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-29922437-2c45-4e63-87a6-73b34556fed0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:57.802276 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29922437-2c45-4e63-87a6-73b34556fed0 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:57.809720 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ec4a1-ad83-4807-bd3f-85ceff1d3ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:57.810854 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ec4a1-ad83-4807-bd3f-85ceff1d3ef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:57.811378 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ec4a1-ad83-4807-bd3f-85ceff1d3ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:57.811725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ec4a1-ad83-4807-bd3f-85ceff1d3ef3 admin admin] PATH_INFO: `/v3/users/403896b926a1445b9895dc2d8d21854f` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:57.813850 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ec4a1-ad83-4807-bd3f-85ceff1d3ef3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=403896b926a1445b9895dc2d8d21854f)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:57.817321 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ec4a1-ad83-4807-bd3f-85ceff1d3ef3 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:57.881828 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-451ec4a1-ad83-4807-bd3f-85ceff1d3ef3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '403896b926a1445b9895dc2d8d21854f'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:57.882507 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-451ec4a1-ad83-4807-bd3f-85ceff1d3ef3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '403896b926a1445b9895dc2d8d21854f'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:57.892893 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-451ec4a1-ad83-4807-bd3f-85ceff1d3ef3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '403896b926a1445b9895dc2d8d21854f'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:57.907292 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-451ec4a1-ad83-4807-bd3f-85ceff1d3ef3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '403896b926a1445b9895dc2d8d21854f'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:57.923514 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-451ec4a1-ad83-4807-bd3f-85ceff1d3ef3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '403896b926a1445b9895dc2d8d21854f'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:57.931178 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1294/2595] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:16:57 2022] DELETE /identity/v3/users/403896b926a1445b9895dc2d8d21854f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:16:58.140588 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6452d7-3e0c-40a7-9fc4-646beb4ef58f service placement] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:58.152599 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6452d7-3e0c-40a7-9fc4-646beb4ef58f service placement] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:58.218612 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ec4a1-ad83-4807-bd3f-85ceff1d3ef3 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:58.220135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-451ec4a1-ad83-4807-bd3f-85ceff1d3ef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:58.239877 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c47f14-fbda-4ba8-92cc-b1773718fe04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:58.241543 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c47f14-fbda-4ba8-92cc-b1773718fe04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:58.241839 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c47f14-fbda-4ba8-92cc-b1773718fe04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:58.242212 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c47f14-fbda-4ba8-92cc-b1773718fe04 admin admin] PATH_INFO: `/v3/projects/c86a28aa09db413593b6047d4cbd0559` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:58.243641 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c47f14-fbda-4ba8-92cc-b1773718fe04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c86a28aa09db413593b6047d4cbd0559)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:58.246118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c47f14-fbda-4ba8-92cc-b1773718fe04 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:58.248354 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:16:58.253023 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:16:58.298552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ec4a1-ad83-4807-bd3f-85ceff1d3ef3 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:58.312302 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-42c47f14-fbda-4ba8-92cc-b1773718fe04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c86a28aa09db413593b6047d4cbd0559', 'request_id': 'req-42c47f14-fbda-4ba8-92cc-b1773718fe04'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:58.318537 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-42c47f14-fbda-4ba8-92cc-b1773718fe04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c86a28aa09db413593b6047d4cbd0559', 'request_id': 'req-42c47f14-fbda-4ba8-92cc-b1773718fe04'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:58.328192 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-42c47f14-fbda-4ba8-92cc-b1773718fe04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c86a28aa09db413593b6047d4cbd0559', 'request_id': 'req-42c47f14-fbda-4ba8-92cc-b1773718fe04'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:58.342118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-42c47f14-fbda-4ba8-92cc-b1773718fe04 admin admin] The token cache is being invalidate because project c86a28aa09db413593b6047d4cbd0559 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:16:58.342398 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-42c47f14-fbda-4ba8-92cc-b1773718fe04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:58.343124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-42c47f14-fbda-4ba8-92cc-b1773718fe04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:58.344939 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1302/2596] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:16:58 2022] DELETE /identity/v3/projects/c86a28aa09db413593b6047d4cbd0559 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:16:58.351597 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df8f77-26b7-4157-9bc1-02336e231d41 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:58.351997 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df8f77-26b7-4157-9bc1-02336e231d41 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:58.352471 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df8f77-26b7-4157-9bc1-02336e231d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:58.361342 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e304c-55cd-4247-8903-ec85665dacad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:58.361994 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e304c-55cd-4247-8903-ec85665dacad service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:58.362278 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e304c-55cd-4247-8903-ec85665dacad service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:58.362552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e304c-55cd-4247-8903-ec85665dacad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:58.364641 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1a8e304c-55cd-4247-8903-ec85665dacad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:58.372529 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-15df8f77-26b7-4157-9bc1-02336e231d41 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:16:58.373768 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-15df8f77-26b7-4157-9bc1-02336e231d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:58.440232 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e304c-55cd-4247-8903-ec85665dacad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:58.441638 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e304c-55cd-4247-8903-ec85665dacad service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:58.453173 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1295/2597] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:16:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:16:58.461422 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1303/2598] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:16:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:16:58.472873 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6002ba-47f8-4cef-82b0-d0194bea92c0 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:58.473158 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6002ba-47f8-4cef-82b0-d0194bea92c0 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:58.473428 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6002ba-47f8-4cef-82b0-d0194bea92c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:58.503745 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-fb6002ba-47f8-4cef-82b0-d0194bea92c0 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:16:58.505915 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-fb6002ba-47f8-4cef-82b0-d0194bea92c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:58.520168 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1296/2599] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:16:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:16:58.521131 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15df8f77-26b7-4157-9bc1-02336e231d41 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:58.533008 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15df8f77-26b7-4157-9bc1-02336e231d41 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:58.538861 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c2e36-c662-4ba6-8e52-e69a820eebe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:58.539796 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c2e36-c662-4ba6-8e52-e69a820eebe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:58.539796 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c2e36-c662-4ba6-8e52-e69a820eebe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:58.540234 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c2e36-c662-4ba6-8e52-e69a820eebe9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:58.540852 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c2e36-c662-4ba6-8e52-e69a820eebe9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:16:58.541185 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c2e36-c662-4ba6-8e52-e69a820eebe9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:58.543222 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c2e36-c662-4ba6-8e52-e69a820eebe9 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:58.561743 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1304/2600] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:16:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:16:58.566495 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6002ba-47f8-4cef-82b0-d0194bea92c0 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:58.576946 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6002ba-47f8-4cef-82b0-d0194bea92c0 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:58.582401 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94127eb1-ad34-4a53-a963-dc2ea2aa682b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:58.583063 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94127eb1-ad34-4a53-a963-dc2ea2aa682b admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:58.583345 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94127eb1-ad34-4a53-a963-dc2ea2aa682b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:58.583618 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94127eb1-ad34-4a53-a963-dc2ea2aa682b admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:58.584328 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94127eb1-ad34-4a53-a963-dc2ea2aa682b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:58.586443 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94127eb1-ad34-4a53-a963-dc2ea2aa682b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:58.628899 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1297/2601] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:16:58 2022] POST /identity/v3/projects => generated 405 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:16:58.630175 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c2e36-c662-4ba6-8e52-e69a820eebe9 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:58.642544 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c2e36-c662-4ba6-8e52-e69a820eebe9 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:58.718482 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22537569-2528-42a1-af7b-514982eca3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:58.719187 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22537569-2528-42a1-af7b-514982eca3d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:58.719862 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22537569-2528-42a1-af7b-514982eca3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:58.719862 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22537569-2528-42a1-af7b-514982eca3d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:58.720332 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22537569-2528-42a1-af7b-514982eca3d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:58.722265 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22537569-2528-42a1-af7b-514982eca3d2 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:58.745553 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1305/2602] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:16:58 2022] POST /identity/v3/users => generated 303 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:16:58.749807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94127eb1-ad34-4a53-a963-dc2ea2aa682b admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:58.760586 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94127eb1-ad34-4a53-a963-dc2ea2aa682b admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:58.766373 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f06d2d-e867-467c-b4ed-e2df8e95a3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:58.767014 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f06d2d-e867-467c-b4ed-e2df8e95a3b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:58.767297 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f06d2d-e867-467c-b4ed-e2df8e95a3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:58.767572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f06d2d-e867-467c-b4ed-e2df8e95a3b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:58.768315 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f06d2d-e867-467c-b4ed-e2df8e95a3b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:58.769984 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f06d2d-e867-467c-b4ed-e2df8e95a3b4 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:58.781188 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1298/2603] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:16:58.785720 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22537569-2528-42a1-af7b-514982eca3d2 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:58.797016 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22537569-2528-42a1-af7b-514982eca3d2 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:58.802172 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c126399-f8e7-4f1b-aed8-a1512cc5f4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:58.802796 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c126399-f8e7-4f1b-aed8-a1512cc5f4d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:58.803074 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c126399-f8e7-4f1b-aed8-a1512cc5f4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:58.803560 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c126399-f8e7-4f1b-aed8-a1512cc5f4d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:58.804257 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c126399-f8e7-4f1b-aed8-a1512cc5f4d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:58.805917 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c126399-f8e7-4f1b-aed8-a1512cc5f4d5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:58.807420 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:16:58.819726 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1306/2604] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:16:58.823905 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f06d2d-e867-467c-b4ed-e2df8e95a3b4 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:58.834089 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f06d2d-e867-467c-b4ed-e2df8e95a3b4 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:58.839358 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88165191-91ac-45bc-893b-18611875bba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:58.840039 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88165191-91ac-45bc-893b-18611875bba3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:58.840436 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88165191-91ac-45bc-893b-18611875bba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:58.840646 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88165191-91ac-45bc-893b-18611875bba3 admin admin] PATH_INFO: `/v3/projects/3204889fa318491bbf836dd5fb38453d/users/41ee04f5f66e4a12a293546f8b36c4e0/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:58.856311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88165191-91ac-45bc-893b-18611875bba3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3204889fa318491bbf836dd5fb38453d, user_id=41ee04f5f66e4a12a293546f8b36c4e0, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:58.859003 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88165191-91ac-45bc-893b-18611875bba3 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:58.873402 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1299/2605] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:16:58 2022] PUT /identity/v3/projects/3204889fa318491bbf836dd5fb38453d/users/41ee04f5f66e4a12a293546f8b36c4e0/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:16:58.878470 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044a89f-b5e4-480b-9d0e-3a2aa75edb5c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:58.878896 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044a89f-b5e4-480b-9d0e-3a2aa75edb5c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:58.879169 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044a89f-b5e4-480b-9d0e-3a2aa75edb5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:58.901197 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-0044a89f-b5e4-480b-9d0e-3a2aa75edb5c None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:16:58.902515 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0044a89f-b5e4-480b-9d0e-3a2aa75edb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:58.966902 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1307/2606] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:16:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:16:58.977063 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88165191-91ac-45bc-893b-18611875bba3 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:58.978834 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-88165191-91ac-45bc-893b-18611875bba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:59.014894 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:16:59.044022 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88165191-91ac-45bc-893b-18611875bba3 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:59.047984 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d998b-bfc7-468a-9ec5-29d313baf1ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:59.048662 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d998b-bfc7-468a-9ec5-29d313baf1ff service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:59.048938 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d998b-bfc7-468a-9ec5-29d313baf1ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:59.049212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d998b-bfc7-468a-9ec5-29d313baf1ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:59.059204 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27d998b-bfc7-468a-9ec5-29d313baf1ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:59.060728 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27d998b-bfc7-468a-9ec5-29d313baf1ff service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:59.071464 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1300/2607] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:16:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:16:59.960098 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88248e51-521c-4272-a096-94c0f6bfc965 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:59.960697 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88248e51-521c-4272-a096-94c0f6bfc965 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:59.960799 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88248e51-521c-4272-a096-94c0f6bfc965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:59.996979 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-88248e51-521c-4272-a096-94c0f6bfc965 None None] MFA Rules not processed for user `41ee04f5f66e4a12a293546f8b36c4e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:16:59.999100 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-88248e51-521c-4272-a096-94c0f6bfc965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:00.120601 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1308/2608] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:16:59 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:17:00.127652 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d998b-bfc7-468a-9ec5-29d313baf1ff service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:00.142592 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d998b-bfc7-468a-9ec5-29d313baf1ff service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:00.149760 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d4a69-a729-4c90-932e-1781f9c547a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:00.150745 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d4a69-a729-4c90-932e-1781f9c547a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:00.151206 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d4a69-a729-4c90-932e-1781f9c547a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:00.151669 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d4a69-a729-4c90-932e-1781f9c547a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:00.152846 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d4a69-a729-4c90-932e-1781f9c547a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:00.156243 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d4a69-a729-4c90-932e-1781f9c547a6 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:00.196339 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1301/2609] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:17:00 2022] POST /identity/v3/projects => generated 405 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:17:00.200256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88248e51-521c-4272-a096-94c0f6bfc965 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:00.212700 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88248e51-521c-4272-a096-94c0f6bfc965 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:00.289272 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54afd31b-067f-44d1-9736-25175d3a8351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:00.289908 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54afd31b-067f-44d1-9736-25175d3a8351 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:00.290172 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54afd31b-067f-44d1-9736-25175d3a8351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:00.290461 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54afd31b-067f-44d1-9736-25175d3a8351 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:00.291140 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54afd31b-067f-44d1-9736-25175d3a8351 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:00.293164 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54afd31b-067f-44d1-9736-25175d3a8351 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:00.318401 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1309/2610] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:17:00 2022] POST /identity/v3/users => generated 309 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:17:00.323188 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d4a69-a729-4c90-932e-1781f9c547a6 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:00.335670 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d4a69-a729-4c90-932e-1781f9c547a6 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:00.340757 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b36338-ad83-4d42-b44c-b1d5e0b856e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:00.341951 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b36338-ad83-4d42-b44c-b1d5e0b856e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:00.342223 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b36338-ad83-4d42-b44c-b1d5e0b856e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:00.342499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b36338-ad83-4d42-b44c-b1d5e0b856e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:00.343227 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b36338-ad83-4d42-b44c-b1d5e0b856e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:00.344876 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b36338-ad83-4d42-b44c-b1d5e0b856e9 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:00.356808 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1302/2611] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:17:00.361339 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54afd31b-067f-44d1-9736-25175d3a8351 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:00.372857 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54afd31b-067f-44d1-9736-25175d3a8351 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:00.379130 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef4bb0e-3158-4a0a-a9d5-9acc7dd87e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:00.379912 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4bb0e-3158-4a0a-a9d5-9acc7dd87e5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:00.380264 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4bb0e-3158-4a0a-a9d5-9acc7dd87e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:00.380646 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4bb0e-3158-4a0a-a9d5-9acc7dd87e5c admin admin] PATH_INFO: `/v3/projects/124520c1ae214c2fbe3e69d86c8b73ab/users/52390772d8fe4a3585009615a70cb286/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:00.395367 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:00.399547 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef4bb0e-3158-4a0a-a9d5-9acc7dd87e5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=124520c1ae214c2fbe3e69d86c8b73ab, user_id=52390772d8fe4a3585009615a70cb286, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:00.402847 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef4bb0e-3158-4a0a-a9d5-9acc7dd87e5c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:00.418468 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1310/2612] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:17:00 2022] PUT /identity/v3/projects/124520c1ae214c2fbe3e69d86c8b73ab/users/52390772d8fe4a3585009615a70cb286/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:17:00.422806 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b36338-ad83-4d42-b44c-b1d5e0b856e9 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:00.433936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b36338-ad83-4d42-b44c-b1d5e0b856e9 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:00.505819 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5470fb-7e59-48bd-8c64-003959d2cd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:00.506549 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5470fb-7e59-48bd-8c64-003959d2cd51 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:00.506806 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5470fb-7e59-48bd-8c64-003959d2cd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:00.507056 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5470fb-7e59-48bd-8c64-003959d2cd51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:00.507757 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5470fb-7e59-48bd-8c64-003959d2cd51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:00.509719 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5470fb-7e59-48bd-8c64-003959d2cd51 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:00.524315 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1303/2613] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:17:00.526434 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef4bb0e-3158-4a0a-a9d5-9acc7dd87e5c admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:00.540424 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef4bb0e-3158-4a0a-a9d5-9acc7dd87e5c admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:00.546913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ade5e2-0a99-45e4-8fda-1a6bc672f217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:00.547747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ade5e2-0a99-45e4-8fda-1a6bc672f217 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:00.549392 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ade5e2-0a99-45e4-8fda-1a6bc672f217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:00.549392 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ade5e2-0a99-45e4-8fda-1a6bc672f217 admin admin] PATH_INFO: `/v3/projects/124520c1ae214c2fbe3e69d86c8b73ab/users/52390772d8fe4a3585009615a70cb286/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:00.553757 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ade5e2-0a99-45e4-8fda-1a6bc672f217 admin admin] RBAC: Authorizing `identity:create_grant(project_id=124520c1ae214c2fbe3e69d86c8b73ab, user_id=52390772d8fe4a3585009615a70cb286, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:00.556867 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ade5e2-0a99-45e4-8fda-1a6bc672f217 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:00.570729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1311/2614] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:17:00 2022] PUT /identity/v3/projects/124520c1ae214c2fbe3e69d86c8b73ab/users/52390772d8fe4a3585009615a70cb286/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:17:01.803430 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177dd01-df7a-4c94-add2-c69a9ad694b4 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:01.804235 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177dd01-df7a-4c94-add2-c69a9ad694b4 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:01.804235 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177dd01-df7a-4c94-add2-c69a9ad694b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:01.836839 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-c177dd01-df7a-4c94-add2-c69a9ad694b4 None None] MFA Rules not processed for user `52390772d8fe4a3585009615a70cb286`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:17:01.838102 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c177dd01-df7a-4c94-add2-c69a9ad694b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:01.960944 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1304/2615] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:17:01 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:17:03.122658 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ade5e2-0a99-45e4-8fda-1a6bc672f217 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:03.219790 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ade5e2-0a99-45e4-8fda-1a6bc672f217 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:03.227341 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04381c5a-f638-4838-bc74-bd0f86d371d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:03.228353 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04381c5a-f638-4838-bc74-bd0f86d371d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:03.228664 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04381c5a-f638-4838-bc74-bd0f86d371d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:03.229031 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04381c5a-f638-4838-bc74-bd0f86d371d4 admin admin] PATH_INFO: `/v3/users/41ee04f5f66e4a12a293546f8b36c4e0` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:03.231038 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04381c5a-f638-4838-bc74-bd0f86d371d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41ee04f5f66e4a12a293546f8b36c4e0)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:03.235080 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04381c5a-f638-4838-bc74-bd0f86d371d4 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:03.306715 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-04381c5a-f638-4838-bc74-bd0f86d371d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41ee04f5f66e4a12a293546f8b36c4e0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:03.307525 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-04381c5a-f638-4838-bc74-bd0f86d371d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41ee04f5f66e4a12a293546f8b36c4e0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:03.320748 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-04381c5a-f638-4838-bc74-bd0f86d371d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41ee04f5f66e4a12a293546f8b36c4e0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:03.328017 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-04381c5a-f638-4838-bc74-bd0f86d371d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41ee04f5f66e4a12a293546f8b36c4e0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:03.328777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-04381c5a-f638-4838-bc74-bd0f86d371d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41ee04f5f66e4a12a293546f8b36c4e0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:03.342562 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1312/2616] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:17:03 2022] DELETE /identity/v3/users/41ee04f5f66e4a12a293546f8b36c4e0 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:17:03.530095 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177dd01-df7a-4c94-add2-c69a9ad694b4 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:03.547690 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177dd01-df7a-4c94-add2-c69a9ad694b4 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:03.567030 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:03.609547 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e909414d-b078-4e98-8298-72261e6352c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:03.610203 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909414d-b078-4e98-8298-72261e6352c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:03.610485 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909414d-b078-4e98-8298-72261e6352c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:03.610757 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909414d-b078-4e98-8298-72261e6352c1 admin admin] PATH_INFO: `/v3/projects/3204889fa318491bbf836dd5fb38453d` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:03.611923 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e909414d-b078-4e98-8298-72261e6352c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3204889fa318491bbf836dd5fb38453d)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:03.613915 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e909414d-b078-4e98-8298-72261e6352c1 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:03.666304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e909414d-b078-4e98-8298-72261e6352c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3204889fa318491bbf836dd5fb38453d', 'request_id': 'req-e909414d-b078-4e98-8298-72261e6352c1'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:03.673828 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e909414d-b078-4e98-8298-72261e6352c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3204889fa318491bbf836dd5fb38453d', 'request_id': 'req-e909414d-b078-4e98-8298-72261e6352c1'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:03.688755 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e909414d-b078-4e98-8298-72261e6352c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3204889fa318491bbf836dd5fb38453d', 'request_id': 'req-e909414d-b078-4e98-8298-72261e6352c1'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:03.700113 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e909414d-b078-4e98-8298-72261e6352c1 admin admin] The token cache is being invalidate because project 3204889fa318491bbf836dd5fb38453d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:17:03.700444 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e909414d-b078-4e98-8298-72261e6352c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:03.701147 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e909414d-b078-4e98-8298-72261e6352c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:03.702937 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1305/2617] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:17:03 2022] DELETE /identity/v3/projects/3204889fa318491bbf836dd5fb38453d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:17:03.708592 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04381c5a-f638-4838-bc74-bd0f86d371d4 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:03.711041 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-04381c5a-f638-4838-bc74-bd0f86d371d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:03.726551 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04381c5a-f638-4838-bc74-bd0f86d371d4 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:03.780194 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:03.787808 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff0366-9aee-428e-a8db-c2b45e3b33da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:03.788556 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff0366-9aee-428e-a8db-c2b45e3b33da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:03.789218 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff0366-9aee-428e-a8db-c2b45e3b33da admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:03.789831 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff0366-9aee-428e-a8db-c2b45e3b33da admin admin] PATH_INFO: `/v3/users/52390772d8fe4a3585009615a70cb286` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:03.791725 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ff0366-9aee-428e-a8db-c2b45e3b33da admin admin] RBAC: Authorizing `identity:delete_user(user_id=52390772d8fe4a3585009615a70cb286)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:03.793963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ff0366-9aee-428e-a8db-c2b45e3b33da admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:03.848187 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-18ff0366-9aee-428e-a8db-c2b45e3b33da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52390772d8fe4a3585009615a70cb286'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:03.848944 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-18ff0366-9aee-428e-a8db-c2b45e3b33da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52390772d8fe4a3585009615a70cb286'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:03.874874 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-18ff0366-9aee-428e-a8db-c2b45e3b33da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52390772d8fe4a3585009615a70cb286'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:03.885285 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-18ff0366-9aee-428e-a8db-c2b45e3b33da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52390772d8fe4a3585009615a70cb286'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:03.885873 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-18ff0366-9aee-428e-a8db-c2b45e3b33da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52390772d8fe4a3585009615a70cb286'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:03.912211 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1313/2618] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:17:03 2022] DELETE /identity/v3/users/52390772d8fe4a3585009615a70cb286 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:17:03.916635 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e909414d-b078-4e98-8298-72261e6352c1 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:03.918087 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e909414d-b078-4e98-8298-72261e6352c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:03.984973 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e909414d-b078-4e98-8298-72261e6352c1 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:04.038221 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c82bac-e6a3-4af3-82e8-597ac6eb61e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:04.039154 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c82bac-e6a3-4af3-82e8-597ac6eb61e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:04.039502 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c82bac-e6a3-4af3-82e8-597ac6eb61e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:04.039845 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c82bac-e6a3-4af3-82e8-597ac6eb61e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:04.041955 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e5c82bac-e6a3-4af3-82e8-597ac6eb61e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:04.064308 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:04.099241 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff0366-9aee-428e-a8db-c2b45e3b33da admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:04.100943 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-18ff0366-9aee-428e-a8db-c2b45e3b33da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:04.126102 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c82bac-e6a3-4af3-82e8-597ac6eb61e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:04.127613 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c82bac-e6a3-4af3-82e8-597ac6eb61e2 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:04.138914 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1306/2619] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:17:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:17:04.175912 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff0366-9aee-428e-a8db-c2b45e3b33da admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:04.180801 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369147c-2976-40c0-8f14-8059990fcc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:04.181531 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369147c-2976-40c0-8f14-8059990fcc7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:04.181873 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369147c-2976-40c0-8f14-8059990fcc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:04.182153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369147c-2976-40c0-8f14-8059990fcc7f admin admin] PATH_INFO: `/v3/projects/124520c1ae214c2fbe3e69d86c8b73ab` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:04.183334 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7369147c-2976-40c0-8f14-8059990fcc7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=124520c1ae214c2fbe3e69d86c8b73ab)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:04.185317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7369147c-2976-40c0-8f14-8059990fcc7f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:04.240890 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7369147c-2976-40c0-8f14-8059990fcc7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '124520c1ae214c2fbe3e69d86c8b73ab', 'request_id': 'req-7369147c-2976-40c0-8f14-8059990fcc7f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:04.250474 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7369147c-2976-40c0-8f14-8059990fcc7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '124520c1ae214c2fbe3e69d86c8b73ab', 'request_id': 'req-7369147c-2976-40c0-8f14-8059990fcc7f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:04.259981 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7369147c-2976-40c0-8f14-8059990fcc7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '124520c1ae214c2fbe3e69d86c8b73ab', 'request_id': 'req-7369147c-2976-40c0-8f14-8059990fcc7f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:04.273156 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7369147c-2976-40c0-8f14-8059990fcc7f admin admin] The token cache is being invalidate because project 124520c1ae214c2fbe3e69d86c8b73ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:17:04.273380 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7369147c-2976-40c0-8f14-8059990fcc7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:04.274022 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7369147c-2976-40c0-8f14-8059990fcc7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:04.276303 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1314/2620] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:17:04 2022] DELETE /identity/v3/projects/124520c1ae214c2fbe3e69d86c8b73ab => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:17:04.281647 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e09de-eff2-4607-951f-34246a083810 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:04.281938 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e09de-eff2-4607-951f-34246a083810 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:04.282435 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e09de-eff2-4607-951f-34246a083810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:04.304147 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-2e8e09de-eff2-4607-951f-34246a083810 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:17:04.305463 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-2e8e09de-eff2-4607-951f-34246a083810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:04.377547 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1307/2621] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:17:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:17:04.386277 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db0378-f38d-409e-aeba-1dcead881c3c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:04.386560 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db0378-f38d-409e-aeba-1dcead881c3c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:04.386832 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db0378-f38d-409e-aeba-1dcead881c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:04.405135 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:04.412928 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-84db0378-f38d-409e-aeba-1dcead881c3c None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:17:04.414905 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-84db0378-f38d-409e-aeba-1dcead881c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:04.433953 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1315/2622] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:17:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:17:04.439284 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e09de-eff2-4607-951f-34246a083810 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:04.450474 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e09de-eff2-4607-951f-34246a083810 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:04.456772 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1b59a-c61d-469f-95ef-e09bb08ab212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:04.457429 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1b59a-c61d-469f-95ef-e09bb08ab212 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:04.457704 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1b59a-c61d-469f-95ef-e09bb08ab212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:04.457970 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1b59a-c61d-469f-95ef-e09bb08ab212 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:04.458849 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d1b59a-c61d-469f-95ef-e09bb08ab212 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:17:04.459009 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d1b59a-c61d-469f-95ef-e09bb08ab212 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:04.461038 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d1b59a-c61d-469f-95ef-e09bb08ab212 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:04.480632 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1308/2623] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:17:04.482051 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84db0378-f38d-409e-aeba-1dcead881c3c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:04.497281 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84db0378-f38d-409e-aeba-1dcead881c3c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:04.503105 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66b060-6ed1-4439-b3bc-6fe2d6ba231e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:04.503760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66b060-6ed1-4439-b3bc-6fe2d6ba231e admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:04.504040 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66b060-6ed1-4439-b3bc-6fe2d6ba231e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:04.504362 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66b060-6ed1-4439-b3bc-6fe2d6ba231e admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:04.505048 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af66b060-6ed1-4439-b3bc-6fe2d6ba231e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:04.507127 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af66b060-6ed1-4439-b3bc-6fe2d6ba231e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:04.547655 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1316/2624] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:17:04 2022] POST /identity/v3/projects => generated 365 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:17:04.548945 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1b59a-c61d-469f-95ef-e09bb08ab212 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:04.561169 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1b59a-c61d-469f-95ef-e09bb08ab212 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:04.626595 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191bb5b0-4ba6-4989-8bed-fecb7c4666d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:04.627236 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bb5b0-4ba6-4989-8bed-fecb7c4666d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:04.627523 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bb5b0-4ba6-4989-8bed-fecb7c4666d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:04.627792 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bb5b0-4ba6-4989-8bed-fecb7c4666d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:04.628503 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191bb5b0-4ba6-4989-8bed-fecb7c4666d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:04.630738 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191bb5b0-4ba6-4989-8bed-fecb7c4666d3 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:04.664819 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1309/2625] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:17:04 2022] POST /identity/v3/users => generated 283 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:17:04.666209 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66b060-6ed1-4439-b3bc-6fe2d6ba231e admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:04.677325 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66b060-6ed1-4439-b3bc-6fe2d6ba231e admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:04.684507 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740d50a6-196c-49bc-9e4f-916088191ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:04.685190 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d50a6-196c-49bc-9e4f-916088191ee0 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:04.685470 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d50a6-196c-49bc-9e4f-916088191ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:04.685750 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d50a6-196c-49bc-9e4f-916088191ee0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:04.686482 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740d50a6-196c-49bc-9e4f-916088191ee0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:04.688119 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740d50a6-196c-49bc-9e4f-916088191ee0 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:04.703048 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1317/2626] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:17:04.704404 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191bb5b0-4ba6-4989-8bed-fecb7c4666d3 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:04.715825 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191bb5b0-4ba6-4989-8bed-fecb7c4666d3 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:04.720759 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b319e0e-8ea5-4c5f-b914-e06da799ccf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:04.721906 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b319e0e-8ea5-4c5f-b914-e06da799ccf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:04.722343 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b319e0e-8ea5-4c5f-b914-e06da799ccf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:04.722693 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b319e0e-8ea5-4c5f-b914-e06da799ccf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:04.723486 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b319e0e-8ea5-4c5f-b914-e06da799ccf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:04.725365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b319e0e-8ea5-4c5f-b914-e06da799ccf2 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:04.737895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1310/2627] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:17:04.742328 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740d50a6-196c-49bc-9e4f-916088191ee0 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:04.752660 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740d50a6-196c-49bc-9e4f-916088191ee0 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:04.757872 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987d94f-1d39-444c-b81a-41d3de5a3920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:04.758629 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987d94f-1d39-444c-b81a-41d3de5a3920 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:04.759095 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987d94f-1d39-444c-b81a-41d3de5a3920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:04.759531 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987d94f-1d39-444c-b81a-41d3de5a3920 admin admin] PATH_INFO: `/v3/projects/6639ce6a7f5f485e991f346be8eea652/users/1bf1a57154e64cf582e2fe82c06dea84/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:04.779152 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987d94f-1d39-444c-b81a-41d3de5a3920 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6639ce6a7f5f485e991f346be8eea652, user_id=1bf1a57154e64cf582e2fe82c06dea84, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:04.782315 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987d94f-1d39-444c-b81a-41d3de5a3920 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:04.799900 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1318/2628] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:17:04 2022] PUT /identity/v3/projects/6639ce6a7f5f485e991f346be8eea652/users/1bf1a57154e64cf582e2fe82c06dea84/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:17:04.815547 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3910db7-5998-443b-8082-afca27bde7c7 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:04.815987 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3910db7-5998-443b-8082-afca27bde7c7 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:04.816477 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3910db7-5998-443b-8082-afca27bde7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:04.849238 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-a3910db7-5998-443b-8082-afca27bde7c7 None None] MFA Rules not processed for user `1bf1a57154e64cf582e2fe82c06dea84`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:17:04.850889 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a3910db7-5998-443b-8082-afca27bde7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:04.866317 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:04.935744 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1311/2629] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:17:04 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:17:04.940649 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987d94f-1d39-444c-b81a-41d3de5a3920 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:05.020566 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987d94f-1d39-444c-b81a-41d3de5a3920 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:05.027159 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5723ed3-70a0-42a3-b630-605f0280d188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:05.027791 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5723ed3-70a0-42a3-b630-605f0280d188 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:05.028069 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5723ed3-70a0-42a3-b630-605f0280d188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:05.028372 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5723ed3-70a0-42a3-b630-605f0280d188 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:05.029054 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5723ed3-70a0-42a3-b630-605f0280d188 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:05.031050 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5723ed3-70a0-42a3-b630-605f0280d188 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:05.065676 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1319/2630] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:17:04 2022] POST /identity/v3/projects => generated 363 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:17:05.066827 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3910db7-5998-443b-8082-afca27bde7c7 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:05.078635 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3910db7-5998-443b-8082-afca27bde7c7 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:05.139510 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe0ba5-a664-4308-8d03-9e20808389da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:05.140146 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe0ba5-a664-4308-8d03-9e20808389da admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:05.140461 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe0ba5-a664-4308-8d03-9e20808389da admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:05.140738 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe0ba5-a664-4308-8d03-9e20808389da admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:05.141414 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abe0ba5-a664-4308-8d03-9e20808389da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:05.143347 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abe0ba5-a664-4308-8d03-9e20808389da admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:05.169497 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1312/2631] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:17:05 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:17:05.170793 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5723ed3-70a0-42a3-b630-605f0280d188 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:05.187299 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5723ed3-70a0-42a3-b630-605f0280d188 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:05.193482 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a75313-836b-476f-905b-332238ec76d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:05.194209 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a75313-836b-476f-905b-332238ec76d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:05.195148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a75313-836b-476f-905b-332238ec76d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:05.195450 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a75313-836b-476f-905b-332238ec76d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:05.196153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a75313-836b-476f-905b-332238ec76d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:05.197886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a75313-836b-476f-905b-332238ec76d5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:05.210258 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1320/2632] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:17:05.215075 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe0ba5-a664-4308-8d03-9e20808389da admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:05.226222 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe0ba5-a664-4308-8d03-9e20808389da admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:05.232441 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533760ae-700d-492a-99f6-71c28e9dcf78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:05.233197 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533760ae-700d-492a-99f6-71c28e9dcf78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:05.233500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533760ae-700d-492a-99f6-71c28e9dcf78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:05.233958 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533760ae-700d-492a-99f6-71c28e9dcf78 admin admin] PATH_INFO: `/v3/projects/51b0c09cbbeb4b66a01cc832483df860/users/526a39a1930841ae8bed798fa291ca6b/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:05.386202 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533760ae-700d-492a-99f6-71c28e9dcf78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51b0c09cbbeb4b66a01cc832483df860, user_id=526a39a1930841ae8bed798fa291ca6b, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:05.389382 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533760ae-700d-492a-99f6-71c28e9dcf78 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:05.406844 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1313/2633] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:17:05 2022] PUT /identity/v3/projects/51b0c09cbbeb4b66a01cc832483df860/users/526a39a1930841ae8bed798fa291ca6b/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:17:05.410946 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a75313-836b-476f-905b-332238ec76d5 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:05.424386 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a75313-836b-476f-905b-332238ec76d5 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:05.493012 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26247b29-4ed8-4cdc-b325-706d3e24dc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:05.493839 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26247b29-4ed8-4cdc-b325-706d3e24dc2c admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:05.494221 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26247b29-4ed8-4cdc-b325-706d3e24dc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:05.494593 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26247b29-4ed8-4cdc-b325-706d3e24dc2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:05.495415 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26247b29-4ed8-4cdc-b325-706d3e24dc2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:05.497503 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26247b29-4ed8-4cdc-b325-706d3e24dc2c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:05.509942 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1321/2634] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:17:05.514150 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533760ae-700d-492a-99f6-71c28e9dcf78 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:05.525266 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533760ae-700d-492a-99f6-71c28e9dcf78 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:05.531572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35382f44-feb1-4ccd-824e-21605693e1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:05.532175 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35382f44-feb1-4ccd-824e-21605693e1d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:05.532500 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35382f44-feb1-4ccd-824e-21605693e1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:05.532777 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35382f44-feb1-4ccd-824e-21605693e1d1 admin admin] PATH_INFO: `/v3/projects/51b0c09cbbeb4b66a01cc832483df860/users/526a39a1930841ae8bed798fa291ca6b/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:05.535258 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35382f44-feb1-4ccd-824e-21605693e1d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51b0c09cbbeb4b66a01cc832483df860, user_id=526a39a1930841ae8bed798fa291ca6b, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:05.538545 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35382f44-feb1-4ccd-824e-21605693e1d1 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:05.553971 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1314/2635] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:17:05 2022] PUT /identity/v3/projects/51b0c09cbbeb4b66a01cc832483df860/users/526a39a1930841ae8bed798fa291ca6b/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:17:05.565064 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1ecc9-b5c2-482a-aee3-121b7559e9ac None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:05.565560 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1ecc9-b5c2-482a-aee3-121b7559e9ac None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:05.565919 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1ecc9-b5c2-482a-aee3-121b7559e9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:05.596992 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:05.601597 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-64e1ecc9-b5c2-482a-aee3-121b7559e9ac None None] MFA Rules not processed for user `526a39a1930841ae8bed798fa291ca6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:17:05.604154 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-64e1ecc9-b5c2-482a-aee3-121b7559e9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:05.695842 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1322/2636] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:17:05 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:17:05.707848 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35382f44-feb1-4ccd-824e-21605693e1d1 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:05.711061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-35382f44-feb1-4ccd-824e-21605693e1d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:05.790352 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:05.802056 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35382f44-feb1-4ccd-824e-21605693e1d1 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:05.806255 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd75e32-38c3-4014-83ea-f3469ca47b0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:05.806941 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd75e32-38c3-4014-83ea-f3469ca47b0c service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:05.807229 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd75e32-38c3-4014-83ea-f3469ca47b0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:05.807537 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd75e32-38c3-4014-83ea-f3469ca47b0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:05.817605 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd75e32-38c3-4014-83ea-f3469ca47b0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:05.819216 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd75e32-38c3-4014-83ea-f3469ca47b0c service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:05.830097 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1315/2637] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:17:05 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:17:05.922373 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1ecc9-b5c2-482a-aee3-121b7559e9ac None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:05.935788 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1ecc9-b5c2-482a-aee3-121b7559e9ac None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:05.943951 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af39a80-1901-437e-974a-e4d46ebeeab1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:05.944705 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af39a80-1901-437e-974a-e4d46ebeeab1 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:05.944987 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af39a80-1901-437e-974a-e4d46ebeeab1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:05.945257 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af39a80-1901-437e-974a-e4d46ebeeab1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:06.022937 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af39a80-1901-437e-974a-e4d46ebeeab1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:06.024600 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af39a80-1901-437e-974a-e4d46ebeeab1 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:06.035651 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1323/2638] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:17:05 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:17:07.219378 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd75e32-38c3-4014-83ea-f3469ca47b0c service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:07.223893 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5dd75e32-38c3-4014-83ea-f3469ca47b0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:07.259995 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd75e32-38c3-4014-83ea-f3469ca47b0c service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:07.266032 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037d4e59-7228-4df8-85e3-bc75ba0873fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:07.266840 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037d4e59-7228-4df8-85e3-bc75ba0873fa service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:07.267118 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037d4e59-7228-4df8-85e3-bc75ba0873fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:07.267383 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037d4e59-7228-4df8-85e3-bc75ba0873fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:07.269247 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-037d4e59-7228-4df8-85e3-bc75ba0873fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:07.365432 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037d4e59-7228-4df8-85e3-bc75ba0873fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:07.368357 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037d4e59-7228-4df8-85e3-bc75ba0873fa service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:07.387350 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1316/2639] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:17:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:17:11.466961 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af39a80-1901-437e-974a-e4d46ebeeab1 service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:11.468896 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-9af39a80-1901-437e-974a-e4d46ebeeab1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:11.595460 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af39a80-1901-437e-974a-e4d46ebeeab1 service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:11.601614 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e485448-04ef-4978-9be4-b77a3eb174cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:11.602531 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e485448-04ef-4978-9be4-b77a3eb174cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:11.602969 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e485448-04ef-4978-9be4-b77a3eb174cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:11.603393 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e485448-04ef-4978-9be4-b77a3eb174cc admin admin] PATH_INFO: `/v3/users/b4ae8decd5354231bebc7bfda44d3285` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:11.605292 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e485448-04ef-4978-9be4-b77a3eb174cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4ae8decd5354231bebc7bfda44d3285)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:11.608760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e485448-04ef-4978-9be4-b77a3eb174cc admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:11.688930 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:11.700720 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2e485448-04ef-4978-9be4-b77a3eb174cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4ae8decd5354231bebc7bfda44d3285'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:11.702192 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2e485448-04ef-4978-9be4-b77a3eb174cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4ae8decd5354231bebc7bfda44d3285'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:11.725444 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2e485448-04ef-4978-9be4-b77a3eb174cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4ae8decd5354231bebc7bfda44d3285'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:11.738386 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2e485448-04ef-4978-9be4-b77a3eb174cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4ae8decd5354231bebc7bfda44d3285'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:11.739667 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2e485448-04ef-4978-9be4-b77a3eb174cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4ae8decd5354231bebc7bfda44d3285'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:11.764827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1324/2640] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:17:11 2022] DELETE /identity/v3/users/b4ae8decd5354231bebc7bfda44d3285 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:17:12.098201 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037d4e59-7228-4df8-85e3-bc75ba0873fa service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:12.113693 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037d4e59-7228-4df8-85e3-bc75ba0873fa service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:12.252816 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b879c3b-6afe-4b4e-b18a-3dcbfad83e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:12.253837 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b879c3b-6afe-4b4e-b18a-3dcbfad83e23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:12.254280 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b879c3b-6afe-4b4e-b18a-3dcbfad83e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:12.254773 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b879c3b-6afe-4b4e-b18a-3dcbfad83e23 admin admin] PATH_INFO: `/v3/projects/6bfbca571c4b4199a577322e866dfae9` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:12.257289 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b879c3b-6afe-4b4e-b18a-3dcbfad83e23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bfbca571c4b4199a577322e866dfae9)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:12.260391 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b879c3b-6afe-4b4e-b18a-3dcbfad83e23 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:12.341426 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-7b879c3b-6afe-4b4e-b18a-3dcbfad83e23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bfbca571c4b4199a577322e866dfae9', 'request_id': 'req-7b879c3b-6afe-4b4e-b18a-3dcbfad83e23'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:12.348808 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-7b879c3b-6afe-4b4e-b18a-3dcbfad83e23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bfbca571c4b4199a577322e866dfae9', 'request_id': 'req-7b879c3b-6afe-4b4e-b18a-3dcbfad83e23'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:12.362968 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-7b879c3b-6afe-4b4e-b18a-3dcbfad83e23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bfbca571c4b4199a577322e866dfae9', 'request_id': 'req-7b879c3b-6afe-4b4e-b18a-3dcbfad83e23'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:12.386884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-7b879c3b-6afe-4b4e-b18a-3dcbfad83e23 admin admin] The token cache is being invalidate because project 6bfbca571c4b4199a577322e866dfae9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:17:12.387745 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-7b879c3b-6afe-4b4e-b18a-3dcbfad83e23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:12.389108 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-7b879c3b-6afe-4b4e-b18a-3dcbfad83e23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:12.391497 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1317/2641] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:17:12 2022] DELETE /identity/v3/projects/6bfbca571c4b4199a577322e866dfae9 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:17:12.395976 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e485448-04ef-4978-9be4-b77a3eb174cc admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:12.397608 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2e485448-04ef-4978-9be4-b77a3eb174cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:12.411943 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e485448-04ef-4978-9be4-b77a3eb174cc admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:12.504058 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1022bed3-1d48-4d64-920c-d3154f51ce45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:12.505045 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022bed3-1d48-4d64-920c-d3154f51ce45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:12.505045 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022bed3-1d48-4d64-920c-d3154f51ce45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:12.505524 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022bed3-1d48-4d64-920c-d3154f51ce45 admin admin] PATH_INFO: `/v3/users/c3bcbc78e809494db785342937dc4971` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:12.506652 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1022bed3-1d48-4d64-920c-d3154f51ce45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3bcbc78e809494db785342937dc4971)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:12.508660 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1022bed3-1d48-4d64-920c-d3154f51ce45 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:12.564955 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1022bed3-1d48-4d64-920c-d3154f51ce45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3bcbc78e809494db785342937dc4971'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:12.565797 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1022bed3-1d48-4d64-920c-d3154f51ce45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3bcbc78e809494db785342937dc4971'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:12.580596 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1022bed3-1d48-4d64-920c-d3154f51ce45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3bcbc78e809494db785342937dc4971'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:12.588343 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1022bed3-1d48-4d64-920c-d3154f51ce45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3bcbc78e809494db785342937dc4971'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:12.589109 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-1022bed3-1d48-4d64-920c-d3154f51ce45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3bcbc78e809494db785342937dc4971'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:12.611706 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1325/2642] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:17:12 2022] DELETE /identity/v3/users/c3bcbc78e809494db785342937dc4971 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:17:12.800463 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b879c3b-6afe-4b4e-b18a-3dcbfad83e23 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:12.801974 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7b879c3b-6afe-4b4e-b18a-3dcbfad83e23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:12.815814 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b879c3b-6afe-4b4e-b18a-3dcbfad83e23 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:12.830855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:12.884830 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c197a4-6243-476e-aabe-a90fcad3439b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:12.885511 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c197a4-6243-476e-aabe-a90fcad3439b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:12.885852 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c197a4-6243-476e-aabe-a90fcad3439b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:12.886047 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c197a4-6243-476e-aabe-a90fcad3439b admin admin] PATH_INFO: `/v3/projects/bfa846b417314968807692fa953e79ad` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:12.887353 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c197a4-6243-476e-aabe-a90fcad3439b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfa846b417314968807692fa953e79ad)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:12.889328 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c197a4-6243-476e-aabe-a90fcad3439b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:12.943811 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a6c197a4-6243-476e-aabe-a90fcad3439b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfa846b417314968807692fa953e79ad', 'request_id': 'req-a6c197a4-6243-476e-aabe-a90fcad3439b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:12.951362 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a6c197a4-6243-476e-aabe-a90fcad3439b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfa846b417314968807692fa953e79ad', 'request_id': 'req-a6c197a4-6243-476e-aabe-a90fcad3439b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:12.961039 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a6c197a4-6243-476e-aabe-a90fcad3439b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfa846b417314968807692fa953e79ad', 'request_id': 'req-a6c197a4-6243-476e-aabe-a90fcad3439b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:12.974230 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a6c197a4-6243-476e-aabe-a90fcad3439b admin admin] The token cache is being invalidate because project bfa846b417314968807692fa953e79ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:17:12.974631 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a6c197a4-6243-476e-aabe-a90fcad3439b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:12.975429 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a6c197a4-6243-476e-aabe-a90fcad3439b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:12.977166 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1318/2643] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:17:12 2022] DELETE /identity/v3/projects/bfa846b417314968807692fa953e79ad => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:17:18.024596 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94dd03-88eb-4d36-9a21-ac200f50205d None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:18.025098 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94dd03-88eb-4d36-9a21-ac200f50205d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:18.025182 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94dd03-88eb-4d36-9a21-ac200f50205d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:18.059516 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-bc94dd03-88eb-4d36-9a21-ac200f50205d None None] MFA Rules not processed for user `8e30910692b4474092f1585a671e2f41`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:17:18.060926 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-bc94dd03-88eb-4d36-9a21-ac200f50205d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:18.113239 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:18.156331 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1326/2644] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:17:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:17:21.026133 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c197a4-6243-476e-aabe-a90fcad3439b admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:21.027986 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a6c197a4-6243-476e-aabe-a90fcad3439b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:21.127470 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c197a4-6243-476e-aabe-a90fcad3439b admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:21.132578 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601d92d-6fc1-4405-806e-250beaa9f6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:21.133258 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601d92d-6fc1-4405-806e-250beaa9f6fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:21.133534 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601d92d-6fc1-4405-806e-250beaa9f6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:21.133805 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601d92d-6fc1-4405-806e-250beaa9f6fb admin admin] PATH_INFO: `/v3/users/1bf1a57154e64cf582e2fe82c06dea84` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:21.135077 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a601d92d-6fc1-4405-806e-250beaa9f6fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bf1a57154e64cf582e2fe82c06dea84)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:21.137078 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a601d92d-6fc1-4405-806e-250beaa9f6fb admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:21.214748 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a601d92d-6fc1-4405-806e-250beaa9f6fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bf1a57154e64cf582e2fe82c06dea84'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:21.216870 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a601d92d-6fc1-4405-806e-250beaa9f6fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bf1a57154e64cf582e2fe82c06dea84'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:21.221251 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:21.235820 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a601d92d-6fc1-4405-806e-250beaa9f6fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bf1a57154e64cf582e2fe82c06dea84'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:21.249152 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a601d92d-6fc1-4405-806e-250beaa9f6fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bf1a57154e64cf582e2fe82c06dea84'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:21.268697 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a601d92d-6fc1-4405-806e-250beaa9f6fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bf1a57154e64cf582e2fe82c06dea84'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:21.277569 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1319/2645] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:17:21 2022] DELETE /identity/v3/users/1bf1a57154e64cf582e2fe82c06dea84 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:17:21.282509 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab473d4-e98a-4d0a-9fd0-4f7a85327d22 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:21.283387 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab473d4-e98a-4d0a-9fd0-4f7a85327d22 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:21.283387 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab473d4-e98a-4d0a-9fd0-4f7a85327d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:21.310189 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-eab473d4-e98a-4d0a-9fd0-4f7a85327d22 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:17:21.312348 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-eab473d4-e98a-4d0a-9fd0-4f7a85327d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:21.416089 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1327/2646] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:17:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:17:21.431097 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601d92d-6fc1-4405-806e-250beaa9f6fb admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:21.432779 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a601d92d-6fc1-4405-806e-250beaa9f6fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:21.536717 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601d92d-6fc1-4405-806e-250beaa9f6fb admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:21.548378 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b12a3-bf6e-4651-a719-fd57041bbe2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:21.549976 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b12a3-bf6e-4651-a719-fd57041bbe2d service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:21.550531 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b12a3-bf6e-4651-a719-fd57041bbe2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:21.551093 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b12a3-bf6e-4651-a719-fd57041bbe2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:21.570512 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b12a3-bf6e-4651-a719-fd57041bbe2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:21.573261 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b12a3-bf6e-4651-a719-fd57041bbe2d service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:21.592018 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1320/2647] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:17:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:17:22.467846 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab473d4-e98a-4d0a-9fd0-4f7a85327d22 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:22.469448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-eab473d4-e98a-4d0a-9fd0-4f7a85327d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:22.482797 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab473d4-e98a-4d0a-9fd0-4f7a85327d22 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:22.488190 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e790b0e0-474e-454c-937d-3a5ddd99eb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:22.488924 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790b0e0-474e-454c-937d-3a5ddd99eb74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:22.489201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790b0e0-474e-454c-937d-3a5ddd99eb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:22.489493 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790b0e0-474e-454c-937d-3a5ddd99eb74 admin admin] PATH_INFO: `/v3/projects/6639ce6a7f5f485e991f346be8eea652` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:22.491080 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e790b0e0-474e-454c-937d-3a5ddd99eb74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6639ce6a7f5f485e991f346be8eea652)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:22.494232 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e790b0e0-474e-454c-937d-3a5ddd99eb74 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:22.582322 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e790b0e0-474e-454c-937d-3a5ddd99eb74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6639ce6a7f5f485e991f346be8eea652', 'request_id': 'req-e790b0e0-474e-454c-937d-3a5ddd99eb74'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:22.590628 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e790b0e0-474e-454c-937d-3a5ddd99eb74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6639ce6a7f5f485e991f346be8eea652', 'request_id': 'req-e790b0e0-474e-454c-937d-3a5ddd99eb74'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:22.607502 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e790b0e0-474e-454c-937d-3a5ddd99eb74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6639ce6a7f5f485e991f346be8eea652', 'request_id': 'req-e790b0e0-474e-454c-937d-3a5ddd99eb74'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:22.632108 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e790b0e0-474e-454c-937d-3a5ddd99eb74 admin admin] The token cache is being invalidate because project 6639ce6a7f5f485e991f346be8eea652 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:17:22.632658 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e790b0e0-474e-454c-937d-3a5ddd99eb74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:22.634763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e790b0e0-474e-454c-937d-3a5ddd99eb74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:22.637440 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1328/2648] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:17:22 2022] DELETE /identity/v3/projects/6639ce6a7f5f485e991f346be8eea652 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:17:22.644328 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b12a3-bf6e-4651-a719-fd57041bbe2d service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:22.658930 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b12a3-bf6e-4651-a719-fd57041bbe2d service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:22.748414 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b3aa56-8841-497a-a5a5-99862111259f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:22.749242 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3aa56-8841-497a-a5a5-99862111259f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:22.749532 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3aa56-8841-497a-a5a5-99862111259f admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:22.750550 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3aa56-8841-497a-a5a5-99862111259f admin admin] PATH_INFO: `/v3/users/526a39a1930841ae8bed798fa291ca6b` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:22.751244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b3aa56-8841-497a-a5a5-99862111259f admin admin] RBAC: Authorizing `identity:delete_user(user_id=526a39a1930841ae8bed798fa291ca6b)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:22.753414 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b3aa56-8841-497a-a5a5-99862111259f admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:22.843638 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a3b3aa56-8841-497a-a5a5-99862111259f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '526a39a1930841ae8bed798fa291ca6b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:22.844571 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a3b3aa56-8841-497a-a5a5-99862111259f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '526a39a1930841ae8bed798fa291ca6b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:22.854734 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a3b3aa56-8841-497a-a5a5-99862111259f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '526a39a1930841ae8bed798fa291ca6b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:22.870266 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a3b3aa56-8841-497a-a5a5-99862111259f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '526a39a1930841ae8bed798fa291ca6b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:22.879433 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:22.917028 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a3b3aa56-8841-497a-a5a5-99862111259f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '526a39a1930841ae8bed798fa291ca6b'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:22.931475 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1321/2649] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:17:22 2022] DELETE /identity/v3/users/526a39a1930841ae8bed798fa291ca6b => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:17:23.427330 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e790b0e0-474e-454c-937d-3a5ddd99eb74 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:23.435377 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e790b0e0-474e-454c-937d-3a5ddd99eb74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:23.494755 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e790b0e0-474e-454c-937d-3a5ddd99eb74 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:23.668872 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c2190-c3ff-4ef2-be8c-eae9b6c6b27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:23.670566 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c2190-c3ff-4ef2-be8c-eae9b6c6b27c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:23.671290 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c2190-c3ff-4ef2-be8c-eae9b6c6b27c admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:23.672603 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c2190-c3ff-4ef2-be8c-eae9b6c6b27c admin admin] PATH_INFO: `/v3/projects/51b0c09cbbeb4b66a01cc832483df860` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:23.674661 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c2190-c3ff-4ef2-be8c-eae9b6c6b27c admin admin] RBAC: Authorizing `identity:delete_project(project_id=51b0c09cbbeb4b66a01cc832483df860)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:23.679083 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c2190-c3ff-4ef2-be8c-eae9b6c6b27c admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:23.691418 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:23.780554 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-557c2190-c3ff-4ef2-be8c-eae9b6c6b27c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51b0c09cbbeb4b66a01cc832483df860', 'request_id': 'req-557c2190-c3ff-4ef2-be8c-eae9b6c6b27c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:23.792059 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-557c2190-c3ff-4ef2-be8c-eae9b6c6b27c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51b0c09cbbeb4b66a01cc832483df860', 'request_id': 'req-557c2190-c3ff-4ef2-be8c-eae9b6c6b27c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:23.808522 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-557c2190-c3ff-4ef2-be8c-eae9b6c6b27c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51b0c09cbbeb4b66a01cc832483df860', 'request_id': 'req-557c2190-c3ff-4ef2-be8c-eae9b6c6b27c'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:23.827202 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-557c2190-c3ff-4ef2-be8c-eae9b6c6b27c admin admin] The token cache is being invalidate because project 51b0c09cbbeb4b66a01cc832483df860 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:17:23.827582 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-557c2190-c3ff-4ef2-be8c-eae9b6c6b27c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:23.828425 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-557c2190-c3ff-4ef2-be8c-eae9b6c6b27c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:23.830575 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1329/2650] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:17:23 2022] DELETE /identity/v3/projects/51b0c09cbbeb4b66a01cc832483df860 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:17:23.837494 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db9c7f-757d-44c9-a9cf-5b675d579e19 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:23.838905 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db9c7f-757d-44c9-a9cf-5b675d579e19 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:23.839509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db9c7f-757d-44c9-a9cf-5b675d579e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:23.866148 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-12db9c7f-757d-44c9-a9cf-5b675d579e19 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:17:23.867784 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-12db9c7f-757d-44c9-a9cf-5b675d579e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:23.976566 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1322/2651] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:17:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:17:23.984584 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f33d7b-ab29-4401-8790-e64902dcd18c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:23.984914 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f33d7b-ab29-4401-8790-e64902dcd18c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:23.985413 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f33d7b-ab29-4401-8790-e64902dcd18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:24.010386 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d2f33d7b-ab29-4401-8790-e64902dcd18c None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:17:24.011278 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d2f33d7b-ab29-4401-8790-e64902dcd18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:24.021869 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1330/2652] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:17:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:17:24.026442 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db9c7f-757d-44c9-a9cf-5b675d579e19 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:24.037913 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db9c7f-757d-44c9-a9cf-5b675d579e19 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:24.044021 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f15ac-b145-4b32-9d61-8276cea4d07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:24.045153 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f15ac-b145-4b32-9d61-8276cea4d07b admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:24.045623 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f15ac-b145-4b32-9d61-8276cea4d07b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:24.045974 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f15ac-b145-4b32-9d61-8276cea4d07b admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:24.046865 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f15ac-b145-4b32-9d61-8276cea4d07b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:17:24.050517 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f15ac-b145-4b32-9d61-8276cea4d07b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:24.050517 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f15ac-b145-4b32-9d61-8276cea4d07b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:24.074292 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1323/2653] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:17:24.076135 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f33d7b-ab29-4401-8790-e64902dcd18c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:24.088806 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f33d7b-ab29-4401-8790-e64902dcd18c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:24.094136 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27e234-edb6-435c-b122-a8381bbf766d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:24.095173 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27e234-edb6-435c-b122-a8381bbf766d admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:24.095463 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27e234-edb6-435c-b122-a8381bbf766d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:24.095737 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27e234-edb6-435c-b122-a8381bbf766d admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:24.096721 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc27e234-edb6-435c-b122-a8381bbf766d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:24.101211 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc27e234-edb6-435c-b122-a8381bbf766d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:24.150928 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1331/2654] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:17:24 2022] POST /identity/v3/projects => generated 373 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:17:24.156232 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f15ac-b145-4b32-9d61-8276cea4d07b admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:24.169999 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f15ac-b145-4b32-9d61-8276cea4d07b admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:24.273986 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f364e10-ddc2-4419-b33d-17ffb21907bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:24.275300 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f364e10-ddc2-4419-b33d-17ffb21907bd admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:24.275960 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f364e10-ddc2-4419-b33d-17ffb21907bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:24.276331 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f364e10-ddc2-4419-b33d-17ffb21907bd admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:24.277638 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f364e10-ddc2-4419-b33d-17ffb21907bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:24.281580 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f364e10-ddc2-4419-b33d-17ffb21907bd admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:24.332130 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27e234-edb6-435c-b122-a8381bbf766d admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:24.333245 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1324/2655] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:17:24 2022] POST /identity/v3/users => generated 287 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:17:24.351892 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27e234-edb6-435c-b122-a8381bbf766d admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:24.359407 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f3bc7-96c0-438b-8f27-e82636dc9a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:24.360569 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f3bc7-96c0-438b-8f27-e82636dc9a98 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:24.361515 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f3bc7-96c0-438b-8f27-e82636dc9a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:24.361515 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f3bc7-96c0-438b-8f27-e82636dc9a98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:24.362757 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6f3bc7-96c0-438b-8f27-e82636dc9a98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:24.365836 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6f3bc7-96c0-438b-8f27-e82636dc9a98 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:24.383079 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1332/2656] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:17:24.388097 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f364e10-ddc2-4419-b33d-17ffb21907bd admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:24.400582 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f364e10-ddc2-4419-b33d-17ffb21907bd admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:24.405763 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22434d9-d714-4532-b72e-73ff0b1d1033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:24.406457 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22434d9-d714-4532-b72e-73ff0b1d1033 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:24.407143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22434d9-d714-4532-b72e-73ff0b1d1033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:24.407143 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22434d9-d714-4532-b72e-73ff0b1d1033 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:24.408133 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22434d9-d714-4532-b72e-73ff0b1d1033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:24.410169 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22434d9-d714-4532-b72e-73ff0b1d1033 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:24.432755 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f3bc7-96c0-438b-8f27-e82636dc9a98 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:24.438451 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1325/2657] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:17:24.454414 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f3bc7-96c0-438b-8f27-e82636dc9a98 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:24.475298 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60371a77-3991-4724-a004-bea854b05a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:24.476006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60371a77-3991-4724-a004-bea854b05a72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:24.476527 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60371a77-3991-4724-a004-bea854b05a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:24.476724 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60371a77-3991-4724-a004-bea854b05a72 admin admin] PATH_INFO: `/v3/projects/cd0972e214f74909985c94cad66147d5/users/6c279b44a7494057988e5beca2e0fd57/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:24.503745 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60371a77-3991-4724-a004-bea854b05a72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd0972e214f74909985c94cad66147d5, user_id=6c279b44a7494057988e5beca2e0fd57, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:24.506995 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60371a77-3991-4724-a004-bea854b05a72 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:24.524417 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1333/2658] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:17:24 2022] PUT /identity/v3/projects/cd0972e214f74909985c94cad66147d5/users/6c279b44a7494057988e5beca2e0fd57/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:17:24.538596 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10740032-6546-4068-a9b9-797728fac6f8 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:24.539456 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10740032-6546-4068-a9b9-797728fac6f8 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:24.540064 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10740032-6546-4068-a9b9-797728fac6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:24.578566 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-10740032-6546-4068-a9b9-797728fac6f8 None None] MFA Rules not processed for user `6c279b44a7494057988e5beca2e0fd57`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:17:24.579908 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-10740032-6546-4068-a9b9-797728fac6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:24.647868 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:24.681266 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60371a77-3991-4724-a004-bea854b05a72 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:24.683836 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1326/2659] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:17:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:17:24.761610 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60371a77-3991-4724-a004-bea854b05a72 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:24.769948 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b82d3f-c54d-41cf-bcc4-5c50278071f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:24.771143 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b82d3f-c54d-41cf-bcc4-5c50278071f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:24.771351 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b82d3f-c54d-41cf-bcc4-5c50278071f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:24.774164 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b82d3f-c54d-41cf-bcc4-5c50278071f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:24.778721 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b82d3f-c54d-41cf-bcc4-5c50278071f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:24.779987 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b82d3f-c54d-41cf-bcc4-5c50278071f0 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:24.823152 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1334/2660] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:17:24 2022] POST /identity/v3/projects => generated 373 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:17:24.825207 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10740032-6546-4068-a9b9-797728fac6f8 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:24.848783 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10740032-6546-4068-a9b9-797728fac6f8 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:24.974428 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669fab1-49e3-4f9d-aafd-9ca59d79c9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:24.975197 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669fab1-49e3-4f9d-aafd-9ca59d79c9e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:24.975703 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669fab1-49e3-4f9d-aafd-9ca59d79c9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:24.976078 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669fab1-49e3-4f9d-aafd-9ca59d79c9e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:24.976913 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8669fab1-49e3-4f9d-aafd-9ca59d79c9e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:24.979189 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8669fab1-49e3-4f9d-aafd-9ca59d79c9e5 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:25.022393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1327/2661] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:17:24 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:17:25.024540 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b82d3f-c54d-41cf-bcc4-5c50278071f0 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:25.038520 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b82d3f-c54d-41cf-bcc4-5c50278071f0 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:25.047908 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b3cb9-9a67-44e7-ba80-cda51a915216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:25.048711 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b3cb9-9a67-44e7-ba80-cda51a915216 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:25.048988 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b3cb9-9a67-44e7-ba80-cda51a915216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:25.049326 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b3cb9-9a67-44e7-ba80-cda51a915216 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:25.050234 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b3cb9-9a67-44e7-ba80-cda51a915216 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:25.051950 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b3cb9-9a67-44e7-ba80-cda51a915216 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:25.087498 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1335/2662] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:17:25.089657 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669fab1-49e3-4f9d-aafd-9ca59d79c9e5 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:25.121020 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669fab1-49e3-4f9d-aafd-9ca59d79c9e5 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:25.137076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d6210-6765-4e41-b9ba-d13fc2ef181a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:25.138580 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d6210-6765-4e41-b9ba-d13fc2ef181a admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:25.139391 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d6210-6765-4e41-b9ba-d13fc2ef181a admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:25.140236 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d6210-6765-4e41-b9ba-d13fc2ef181a admin admin] PATH_INFO: `/v3/projects/fe6c8482332647c9b479e65e0954ba3c/users/857b3e55f6e54a0b840c55f660aa74f8/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:25.173154 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01d6210-6765-4e41-b9ba-d13fc2ef181a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe6c8482332647c9b479e65e0954ba3c, user_id=857b3e55f6e54a0b840c55f660aa74f8, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:25.179018 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01d6210-6765-4e41-b9ba-d13fc2ef181a admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:25.208149 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1328/2663] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:17:25 2022] PUT /identity/v3/projects/fe6c8482332647c9b479e65e0954ba3c/users/857b3e55f6e54a0b840c55f660aa74f8/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:17:25.213725 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b3cb9-9a67-44e7-ba80-cda51a915216 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:25.224335 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:25.230109 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b3cb9-9a67-44e7-ba80-cda51a915216 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:25.354771 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ab83d-15c5-40de-8ed1-56b0b3c80e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:25.355620 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ab83d-15c5-40de-8ed1-56b0b3c80e3d admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:25.355896 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ab83d-15c5-40de-8ed1-56b0b3c80e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:25.356175 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ab83d-15c5-40de-8ed1-56b0b3c80e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:25.356938 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ab83d-15c5-40de-8ed1-56b0b3c80e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:25.358684 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ab83d-15c5-40de-8ed1-56b0b3c80e3d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:25.370703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1336/2664] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:17:25.374957 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d6210-6765-4e41-b9ba-d13fc2ef181a admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:25.388622 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d6210-6765-4e41-b9ba-d13fc2ef181a admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:25.395981 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba8dab-7493-45cd-8f9c-242bf9f1016e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:25.396901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba8dab-7493-45cd-8f9c-242bf9f1016e admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:25.397283 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba8dab-7493-45cd-8f9c-242bf9f1016e admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:25.398212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba8dab-7493-45cd-8f9c-242bf9f1016e admin admin] PATH_INFO: `/v3/projects/fe6c8482332647c9b479e65e0954ba3c/users/857b3e55f6e54a0b840c55f660aa74f8/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:25.400299 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daba8dab-7493-45cd-8f9c-242bf9f1016e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe6c8482332647c9b479e65e0954ba3c, user_id=857b3e55f6e54a0b840c55f660aa74f8, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:25.403158 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daba8dab-7493-45cd-8f9c-242bf9f1016e admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:25.428306 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1329/2665] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:17:25 2022] PUT /identity/v3/projects/fe6c8482332647c9b479e65e0954ba3c/users/857b3e55f6e54a0b840c55f660aa74f8/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:17:25.438055 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c065348-113d-44c7-9014-cb7e808e6148 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:25.438456 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c065348-113d-44c7-9014-cb7e808e6148 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:25.438632 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c065348-113d-44c7-9014-cb7e808e6148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:25.472907 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-4c065348-113d-44c7-9014-cb7e808e6148 None None] MFA Rules not processed for user `857b3e55f6e54a0b840c55f660aa74f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:17:25.473798 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4c065348-113d-44c7-9014-cb7e808e6148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:25.580984 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1337/2666] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:17:25 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:17:25.593603 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba8dab-7493-45cd-8f9c-242bf9f1016e admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:25.595772 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-daba8dab-7493-45cd-8f9c-242bf9f1016e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:25.680252 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba8dab-7493-45cd-8f9c-242bf9f1016e admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:25.685016 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ada0b-c767-468c-a880-77bca12a8d91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:25.685742 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ada0b-c767-468c-a880-77bca12a8d91 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:25.686061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ada0b-c767-468c-a880-77bca12a8d91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:25.686298 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ada0b-c767-468c-a880-77bca12a8d91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:25.777101 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:25.796198 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ada0b-c767-468c-a880-77bca12a8d91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:25.799730 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ada0b-c767-468c-a880-77bca12a8d91 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:25.818568 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1330/2667] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:17:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:17:27.615487 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef22e3a-5f92-4037-8424-9da7a3a7363a None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:27.616296 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef22e3a-5f92-4037-8424-9da7a3a7363a None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:27.616296 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef22e3a-5f92-4037-8424-9da7a3a7363a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:27.639046 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-fef22e3a-5f92-4037-8424-9da7a3a7363a None None] MFA Rules not processed for user `a9fc33bdca304c33b6a78122a54f492a`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:17:27.640564 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fef22e3a-5f92-4037-8424-9da7a3a7363a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:27.720447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1338/2668] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:17:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:17:31.867410 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ada0b-c767-468c-a880-77bca12a8d91 service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:31.869041 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c66ada0b-c767-468c-a880-77bca12a8d91 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:31.881596 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ada0b-c767-468c-a880-77bca12a8d91 service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:31.885506 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c2fe90-5e10-4fb0-be78-ad78bcbf54ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:31.886128 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2fe90-5e10-4fb0-be78-ad78bcbf54ea service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:31.886405 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2fe90-5e10-4fb0-be78-ad78bcbf54ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:31.886672 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2fe90-5e10-4fb0-be78-ad78bcbf54ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:31.898271 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c2fe90-5e10-4fb0-be78-ad78bcbf54ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:31.899773 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c2fe90-5e10-4fb0-be78-ad78bcbf54ea service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:31.910901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1331/2669] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:17:31 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:17:37.714541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=86 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:41.905701 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=11 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:43.955673 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef22e3a-5f92-4037-8424-9da7a3a7363a None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:43.957469 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fef22e3a-5f92-4037-8424-9da7a3a7363a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:44.022467 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef22e3a-5f92-4037-8424-9da7a3a7363a None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:44.026391 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0c870-de0c-40fe-8e84-54c94773be59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:44.027032 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0c870-de0c-40fe-8e84-54c94773be59 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:44.027383 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0c870-de0c-40fe-8e84-54c94773be59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:44.027659 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0c870-de0c-40fe-8e84-54c94773be59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:44.029674 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-41e0c870-de0c-40fe-8e84-54c94773be59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:44.099361 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e0c870-de0c-40fe-8e84-54c94773be59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:44.100855 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e0c870-de0c-40fe-8e84-54c94773be59 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:44.111207 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1339/2670] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:17:43 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:17:54.106520 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=50 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:57.279411 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c2fe90-5e10-4fb0-be78-ad78bcbf54ea service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:57.344007 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c2fe90-5e10-4fb0-be78-ad78bcbf54ea service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:57.349412 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81617094-b6f9-463e-abf2-2e111a5920e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:57.350095 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81617094-b6f9-463e-abf2-2e111a5920e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:57.350365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81617094-b6f9-463e-abf2-2e111a5920e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:57.350634 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81617094-b6f9-463e-abf2-2e111a5920e9 admin admin] PATH_INFO: `/v3/users/6c279b44a7494057988e5beca2e0fd57` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:57.351938 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81617094-b6f9-463e-abf2-2e111a5920e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c279b44a7494057988e5beca2e0fd57)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:57.353963 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81617094-b6f9-463e-abf2-2e111a5920e9 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:57.404504 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-81617094-b6f9-463e-abf2-2e111a5920e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c279b44a7494057988e5beca2e0fd57'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:57.405072 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-81617094-b6f9-463e-abf2-2e111a5920e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c279b44a7494057988e5beca2e0fd57'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:57.414468 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-81617094-b6f9-463e-abf2-2e111a5920e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c279b44a7494057988e5beca2e0fd57'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:57.421801 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-81617094-b6f9-463e-abf2-2e111a5920e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c279b44a7494057988e5beca2e0fd57'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:57.433514 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-81617094-b6f9-463e-abf2-2e111a5920e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c279b44a7494057988e5beca2e0fd57'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:57.439600 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1332/2671] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:17:57 2022] DELETE /identity/v3/users/6c279b44a7494057988e5beca2e0fd57 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:17:57.444499 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb5710-be93-447a-94ac-831696253d68 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:57.445024 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb5710-be93-447a-94ac-831696253d68 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:57.445024 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb5710-be93-447a-94ac-831696253d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:57.465230 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-91cb5710-be93-447a-94ac-831696253d68 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:17:57.466451 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-91cb5710-be93-447a-94ac-831696253d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:57.551649 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1340/2672] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:17:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:17:57.562019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81617094-b6f9-463e-abf2-2e111a5920e9 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:57.563887 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-81617094-b6f9-463e-abf2-2e111a5920e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:57.625235 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81617094-b6f9-463e-abf2-2e111a5920e9 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:57.629418 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c597bc-75bd-4e0c-82ea-8c795bd0eab3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:57.630078 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c597bc-75bd-4e0c-82ea-8c795bd0eab3 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:57.630373 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c597bc-75bd-4e0c-82ea-8c795bd0eab3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:57.630646 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c597bc-75bd-4e0c-82ea-8c795bd0eab3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:57.642682 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c597bc-75bd-4e0c-82ea-8c795bd0eab3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:57.644141 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c597bc-75bd-4e0c-82ea-8c795bd0eab3 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:57.653680 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1333/2673] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:17:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:17:57.960932 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb5710-be93-447a-94ac-831696253d68 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:57.962831 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-91cb5710-be93-447a-94ac-831696253d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:57.981616 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb5710-be93-447a-94ac-831696253d68 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:57.987346 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5863fe-a562-4a78-b9f9-b31b77727e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:57.988022 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5863fe-a562-4a78-b9f9-b31b77727e71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:57.988428 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5863fe-a562-4a78-b9f9-b31b77727e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:57.988693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5863fe-a562-4a78-b9f9-b31b77727e71 admin admin] PATH_INFO: `/v3/projects/cd0972e214f74909985c94cad66147d5` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:57.990026 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5863fe-a562-4a78-b9f9-b31b77727e71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd0972e214f74909985c94cad66147d5)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:57.992084 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5863fe-a562-4a78-b9f9-b31b77727e71 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:58.050856 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6b5863fe-a562-4a78-b9f9-b31b77727e71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd0972e214f74909985c94cad66147d5', 'request_id': 'req-6b5863fe-a562-4a78-b9f9-b31b77727e71'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:58.056825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6b5863fe-a562-4a78-b9f9-b31b77727e71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd0972e214f74909985c94cad66147d5', 'request_id': 'req-6b5863fe-a562-4a78-b9f9-b31b77727e71'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:58.066431 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6b5863fe-a562-4a78-b9f9-b31b77727e71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd0972e214f74909985c94cad66147d5', 'request_id': 'req-6b5863fe-a562-4a78-b9f9-b31b77727e71'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:58.078059 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6b5863fe-a562-4a78-b9f9-b31b77727e71 admin admin] The token cache is being invalidate because project cd0972e214f74909985c94cad66147d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:17:58.078348 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6b5863fe-a562-4a78-b9f9-b31b77727e71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:58.078993 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-6b5863fe-a562-4a78-b9f9-b31b77727e71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:58.080571 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1341/2674] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:17:57 2022] DELETE /identity/v3/projects/cd0972e214f74909985c94cad66147d5 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:17:58.085196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c597bc-75bd-4e0c-82ea-8c795bd0eab3 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:58.094799 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c597bc-75bd-4e0c-82ea-8c795bd0eab3 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:58.113211 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:58.153863 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c3dfb-64c8-4284-9527-8a80fcbbf1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:58.154531 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c3dfb-64c8-4284-9527-8a80fcbbf1d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:58.154809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c3dfb-64c8-4284-9527-8a80fcbbf1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:58.155073 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c3dfb-64c8-4284-9527-8a80fcbbf1d7 admin admin] PATH_INFO: `/v3/users/857b3e55f6e54a0b840c55f660aa74f8` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:58.156351 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c3dfb-64c8-4284-9527-8a80fcbbf1d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=857b3e55f6e54a0b840c55f660aa74f8)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:58.158287 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c3dfb-64c8-4284-9527-8a80fcbbf1d7 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:58.211393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a22c3dfb-64c8-4284-9527-8a80fcbbf1d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '857b3e55f6e54a0b840c55f660aa74f8'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:58.211950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a22c3dfb-64c8-4284-9527-8a80fcbbf1d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '857b3e55f6e54a0b840c55f660aa74f8'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:58.221523 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a22c3dfb-64c8-4284-9527-8a80fcbbf1d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '857b3e55f6e54a0b840c55f660aa74f8'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:58.229860 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a22c3dfb-64c8-4284-9527-8a80fcbbf1d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '857b3e55f6e54a0b840c55f660aa74f8'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:58.245808 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a22c3dfb-64c8-4284-9527-8a80fcbbf1d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '857b3e55f6e54a0b840c55f660aa74f8'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:58.255391 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1334/2675] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:17:58 2022] DELETE /identity/v3/users/857b3e55f6e54a0b840c55f660aa74f8 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:17:58.565274 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5863fe-a562-4a78-b9f9-b31b77727e71 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:58.566806 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-6b5863fe-a562-4a78-b9f9-b31b77727e71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:58.581087 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5863fe-a562-4a78-b9f9-b31b77727e71 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:58.644077 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03402ea-8d96-4c31-b8e7-d0042337c6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:58.644783 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03402ea-8d96-4c31-b8e7-d0042337c6ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:58.645094 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03402ea-8d96-4c31-b8e7-d0042337c6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:58.645407 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03402ea-8d96-4c31-b8e7-d0042337c6ba admin admin] PATH_INFO: `/v3/projects/fe6c8482332647c9b479e65e0954ba3c` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:58.646557 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03402ea-8d96-4c31-b8e7-d0042337c6ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe6c8482332647c9b479e65e0954ba3c)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:58.648486 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03402ea-8d96-4c31-b8e7-d0042337c6ba admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:58.681283 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:58.699071 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e03402ea-8d96-4c31-b8e7-d0042337c6ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe6c8482332647c9b479e65e0954ba3c', 'request_id': 'req-e03402ea-8d96-4c31-b8e7-d0042337c6ba'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:58.704491 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e03402ea-8d96-4c31-b8e7-d0042337c6ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe6c8482332647c9b479e65e0954ba3c', 'request_id': 'req-e03402ea-8d96-4c31-b8e7-d0042337c6ba'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:58.713534 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e03402ea-8d96-4c31-b8e7-d0042337c6ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe6c8482332647c9b479e65e0954ba3c', 'request_id': 'req-e03402ea-8d96-4c31-b8e7-d0042337c6ba'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:58.725424 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e03402ea-8d96-4c31-b8e7-d0042337c6ba admin admin] The token cache is being invalidate because project fe6c8482332647c9b479e65e0954ba3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:17:58.725712 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e03402ea-8d96-4c31-b8e7-d0042337c6ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:58.726475 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e03402ea-8d96-4c31-b8e7-d0042337c6ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:58.728179 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1342/2676] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:17:58 2022] DELETE /identity/v3/projects/fe6c8482332647c9b479e65e0954ba3c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:17:58.733584 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0fb0ea-4950-4f71-826e-db60c2ccb7c2 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:58.734107 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0fb0ea-4950-4f71-826e-db60c2ccb7c2 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:58.734107 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0fb0ea-4950-4f71-826e-db60c2ccb7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:58.754921 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-be0fb0ea-4950-4f71-826e-db60c2ccb7c2 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:17:58.756170 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-be0fb0ea-4950-4f71-826e-db60c2ccb7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:58.817565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1335/2677] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:17:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:17:58.885683 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6d70e-4be8-4de9-9dc5-4f00b6cdfbd8 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:58.886000 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6d70e-4be8-4de9-9dc5-4f00b6cdfbd8 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:58.886261 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6d70e-4be8-4de9-9dc5-4f00b6cdfbd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:58.912158 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-25a6d70e-4be8-4de9-9dc5-4f00b6cdfbd8 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:17:58.913435 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-25a6d70e-4be8-4de9-9dc5-4f00b6cdfbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:58.923511 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1343/2678] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:17:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:17:58.927981 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0fb0ea-4950-4f71-826e-db60c2ccb7c2 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:58.938129 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0fb0ea-4950-4f71-826e-db60c2ccb7c2 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:58.943398 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387a103c-ff8d-48b8-aef4-2d0cebabf079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:58.944021 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a103c-ff8d-48b8-aef4-2d0cebabf079 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:58.944328 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a103c-ff8d-48b8-aef4-2d0cebabf079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:58.944600 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a103c-ff8d-48b8-aef4-2d0cebabf079 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:58.945341 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387a103c-ff8d-48b8-aef4-2d0cebabf079 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:17:58.945641 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387a103c-ff8d-48b8-aef4-2d0cebabf079 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:58.947252 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387a103c-ff8d-48b8-aef4-2d0cebabf079 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:58.959607 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1336/2679] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:17:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:17:58.964257 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a6d70e-4be8-4de9-9dc5-4f00b6cdfbd8 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:58.974478 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a6d70e-4be8-4de9-9dc5-4f00b6cdfbd8 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:58.980010 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbada84-a422-4eee-b77e-9c5442aef1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:58.980614 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbada84-a422-4eee-b77e-9c5442aef1c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:58.980895 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbada84-a422-4eee-b77e-9c5442aef1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:58.981168 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbada84-a422-4eee-b77e-9c5442aef1c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:58.981838 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbada84-a422-4eee-b77e-9c5442aef1c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:58.983861 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbada84-a422-4eee-b77e-9c5442aef1c3 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:59.019534 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1344/2680] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:17:58 2022] POST /identity/v3/projects => generated 391 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:17:59.024285 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387a103c-ff8d-48b8-aef4-2d0cebabf079 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:59.035685 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387a103c-ff8d-48b8-aef4-2d0cebabf079 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:59.065259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:59.100568 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750fc16-ff8e-4e71-bc55-719c783cfda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:59.101201 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750fc16-ff8e-4e71-bc55-719c783cfda4 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:59.101510 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750fc16-ff8e-4e71-bc55-719c783cfda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:59.101784 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750fc16-ff8e-4e71-bc55-719c783cfda4 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:59.102473 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3750fc16-ff8e-4e71-bc55-719c783cfda4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:59.104741 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3750fc16-ff8e-4e71-bc55-719c783cfda4 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:59.129928 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1337/2681] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:17:59 2022] POST /identity/v3/users => generated 296 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:17:59.135394 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbada84-a422-4eee-b77e-9c5442aef1c3 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:59.149080 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbada84-a422-4eee-b77e-9c5442aef1c3 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:59.155207 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef29ac8-4601-4abf-890e-652631298f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:59.156245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef29ac8-4601-4abf-890e-652631298f55 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:59.156769 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef29ac8-4601-4abf-890e-652631298f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:59.157161 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef29ac8-4601-4abf-890e-652631298f55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:59.158028 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef29ac8-4601-4abf-890e-652631298f55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:59.160587 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef29ac8-4601-4abf-890e-652631298f55 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:59.175024 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1345/2682] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:17:59.179900 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750fc16-ff8e-4e71-bc55-719c783cfda4 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:59.194076 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750fc16-ff8e-4e71-bc55-719c783cfda4 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:59.199525 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cddee3-64ec-4476-85c4-4732d67b15a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:59.200153 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cddee3-64ec-4476-85c4-4732d67b15a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:59.200462 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cddee3-64ec-4476-85c4-4732d67b15a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:59.200731 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cddee3-64ec-4476-85c4-4732d67b15a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:59.201426 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cddee3-64ec-4476-85c4-4732d67b15a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:59.203032 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cddee3-64ec-4476-85c4-4732d67b15a8 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:59.214217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1338/2683] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:17:59.218508 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef29ac8-4601-4abf-890e-652631298f55 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:59.230997 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef29ac8-4601-4abf-890e-652631298f55 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:59.237757 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027d2a9-96c7-4b7d-b34e-c211410d6d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:59.238813 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027d2a9-96c7-4b7d-b34e-c211410d6d4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:59.239259 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027d2a9-96c7-4b7d-b34e-c211410d6d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:59.239688 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027d2a9-96c7-4b7d-b34e-c211410d6d4f admin admin] PATH_INFO: `/v3/projects/9d0b425de0a84dfb9018c530ad0ec300/users/43536f5bf2e745bbb6ff9fc84432345c/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:59.259566 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5027d2a9-96c7-4b7d-b34e-c211410d6d4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d0b425de0a84dfb9018c530ad0ec300, user_id=43536f5bf2e745bbb6ff9fc84432345c, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:59.262501 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5027d2a9-96c7-4b7d-b34e-c211410d6d4f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:59.282081 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1346/2684] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:17:59 2022] PUT /identity/v3/projects/9d0b425de0a84dfb9018c530ad0ec300/users/43536f5bf2e745bbb6ff9fc84432345c/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:17:59.294308 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435b543-9f88-4964-bb9a-a87528e72f3b None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:59.294588 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435b543-9f88-4964-bb9a-a87528e72f3b None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:59.294871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435b543-9f88-4964-bb9a-a87528e72f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:59.323098 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-4435b543-9f88-4964-bb9a-a87528e72f3b None None] MFA Rules not processed for user `43536f5bf2e745bbb6ff9fc84432345c`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:17:59.324320 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-4435b543-9f88-4964-bb9a-a87528e72f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:59.395525 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1339/2685] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:17:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:17:59.400397 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027d2a9-96c7-4b7d-b34e-c211410d6d4f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:59.474210 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027d2a9-96c7-4b7d-b34e-c211410d6d4f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:59.479223 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0bdc5-0d58-408a-9230-d35820eb0a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:59.479900 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0bdc5-0d58-408a-9230-d35820eb0a78 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:59.480171 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0bdc5-0d58-408a-9230-d35820eb0a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:59.480474 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0bdc5-0d58-408a-9230-d35820eb0a78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:59.481209 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c0bdc5-0d58-408a-9230-d35820eb0a78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:59.483043 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c0bdc5-0d58-408a-9230-d35820eb0a78 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:59.515670 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1347/2686] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:17:59 2022] POST /identity/v3/projects => generated 393 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:17:59.520585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435b543-9f88-4964-bb9a-a87528e72f3b None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:59.534393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435b543-9f88-4964-bb9a-a87528e72f3b None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:59.598085 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd3fbe-32ad-40b2-a146-816fbccda5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:59.598713 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd3fbe-32ad-40b2-a146-816fbccda5ea admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:59.598995 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd3fbe-32ad-40b2-a146-816fbccda5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:59.599260 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd3fbe-32ad-40b2-a146-816fbccda5ea admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:59.599929 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dd3fbe-32ad-40b2-a146-816fbccda5ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:59.601917 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dd3fbe-32ad-40b2-a146-816fbccda5ea admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:59.625041 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1340/2687] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:17:59 2022] POST /identity/v3/users => generated 303 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:17:59.630901 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0bdc5-0d58-408a-9230-d35820eb0a78 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:59.642509 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0bdc5-0d58-408a-9230-d35820eb0a78 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:59.648074 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355e954-a092-4b6a-8ae4-6c66925a586f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:59.650172 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355e954-a092-4b6a-8ae4-6c66925a586f admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:59.650457 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355e954-a092-4b6a-8ae4-6c66925a586f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:59.650733 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355e954-a092-4b6a-8ae4-6c66925a586f admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:59.651470 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f355e954-a092-4b6a-8ae4-6c66925a586f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:59.653117 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f355e954-a092-4b6a-8ae4-6c66925a586f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:59.664583 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1348/2688] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:17:59.669554 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd3fbe-32ad-40b2-a146-816fbccda5ea admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:59.683411 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd3fbe-32ad-40b2-a146-816fbccda5ea admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:59.690036 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95107aac-e5dc-45ef-8611-2c8bf22a51e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:59.691131 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95107aac-e5dc-45ef-8611-2c8bf22a51e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:59.691608 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95107aac-e5dc-45ef-8611-2c8bf22a51e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:59.692100 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95107aac-e5dc-45ef-8611-2c8bf22a51e2 admin admin] PATH_INFO: `/v3/projects/22d2b657892d4beabc97a91d41d9041a/users/b25b3b869ef4462394ab648cc0ef8b03/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:59.717149 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95107aac-e5dc-45ef-8611-2c8bf22a51e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22d2b657892d4beabc97a91d41d9041a, user_id=b25b3b869ef4462394ab648cc0ef8b03, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:59.721495 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95107aac-e5dc-45ef-8611-2c8bf22a51e2 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:59.739103 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1341/2689] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:17:59 2022] PUT /identity/v3/projects/22d2b657892d4beabc97a91d41d9041a/users/b25b3b869ef4462394ab648cc0ef8b03/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:17:59.743275 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355e954-a092-4b6a-8ae4-6c66925a586f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:59.754251 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355e954-a092-4b6a-8ae4-6c66925a586f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:59.773943 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:59.816007 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafefc55-bd7e-44d3-be65-ceb93a9f5413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:59.816655 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafefc55-bd7e-44d3-be65-ceb93a9f5413 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:59.817064 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafefc55-bd7e-44d3-be65-ceb93a9f5413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:59.817332 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafefc55-bd7e-44d3-be65-ceb93a9f5413 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:59.818022 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafefc55-bd7e-44d3-be65-ceb93a9f5413 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:59.819617 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafefc55-bd7e-44d3-be65-ceb93a9f5413 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:59.831274 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1349/2690] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:17:59.836234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95107aac-e5dc-45ef-8611-2c8bf22a51e2 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:59.842121 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:59.849338 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95107aac-e5dc-45ef-8611-2c8bf22a51e2 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:59.855311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c9b34-770b-4f6f-95d1-ccb165da4de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:59.855986 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c9b34-770b-4f6f-95d1-ccb165da4de4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:59.856265 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c9b34-770b-4f6f-95d1-ccb165da4de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:59.856580 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c9b34-770b-4f6f-95d1-ccb165da4de4 admin admin] PATH_INFO: `/v3/projects/22d2b657892d4beabc97a91d41d9041a/users/b25b3b869ef4462394ab648cc0ef8b03/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:59.858687 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81c9b34-770b-4f6f-95d1-ccb165da4de4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22d2b657892d4beabc97a91d41d9041a, user_id=b25b3b869ef4462394ab648cc0ef8b03, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:59.861380 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81c9b34-770b-4f6f-95d1-ccb165da4de4 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:59.872685 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1342/2691] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:17:59 2022] PUT /identity/v3/projects/22d2b657892d4beabc97a91d41d9041a/users/b25b3b869ef4462394ab648cc0ef8b03/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:17:59.883318 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf3d70-cd7b-4ff0-95f1-dc5b6ae75943 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:59.883603 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf3d70-cd7b-4ff0-95f1-dc5b6ae75943 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:59.883879 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf3d70-cd7b-4ff0-95f1-dc5b6ae75943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:59.912721 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-35cf3d70-cd7b-4ff0-95f1-dc5b6ae75943 None None] MFA Rules not processed for user `b25b3b869ef4462394ab648cc0ef8b03`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:17:59.913933 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-35cf3d70-cd7b-4ff0-95f1-dc5b6ae75943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:59.992367 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1350/2692] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:17:59 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:00.004901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c9b34-770b-4f6f-95d1-ccb165da4de4 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:00.006855 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-d81c9b34-770b-4f6f-95d1-ccb165da4de4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:00.078518 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c9b34-770b-4f6f-95d1-ccb165da4de4 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:00.084034 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a91f8cc-4767-4bf7-820b-111ca02a354f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:00.084702 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91f8cc-4767-4bf7-820b-111ca02a354f service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:00.084984 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91f8cc-4767-4bf7-820b-111ca02a354f service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:00.085260 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91f8cc-4767-4bf7-820b-111ca02a354f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:00.105083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a91f8cc-4767-4bf7-820b-111ca02a354f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:00.108803 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a91f8cc-4767-4bf7-820b-111ca02a354f service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:00.120992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1343/2693] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:18:00.173699 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf3d70-cd7b-4ff0-95f1-dc5b6ae75943 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:00.175637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-35cf3d70-cd7b-4ff0-95f1-dc5b6ae75943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:00.189797 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf3d70-cd7b-4ff0-95f1-dc5b6ae75943 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:00.196308 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47baf0cb-901d-494b-8298-d6be09333019 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:00.197341 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47baf0cb-901d-494b-8298-d6be09333019 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:00.197838 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47baf0cb-901d-494b-8298-d6be09333019 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:00.198290 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47baf0cb-901d-494b-8298-d6be09333019 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:00.270991 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47baf0cb-901d-494b-8298-d6be09333019 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:00.272390 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47baf0cb-901d-494b-8298-d6be09333019 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:00.281893 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1351/2694] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:00 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:18:03.707627 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a91f8cc-4767-4bf7-820b-111ca02a354f service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:03.778393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a91f8cc-4767-4bf7-820b-111ca02a354f service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:03.783489 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74d5e5-2191-4dde-a981-8d951891339e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:03.784149 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74d5e5-2191-4dde-a981-8d951891339e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:03.784829 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74d5e5-2191-4dde-a981-8d951891339e admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:03.785105 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74d5e5-2191-4dde-a981-8d951891339e admin admin] PATH_INFO: `/v3/users/43536f5bf2e745bbb6ff9fc84432345c` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:03.787230 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74d5e5-2191-4dde-a981-8d951891339e admin admin] RBAC: Authorizing `identity:delete_user(user_id=43536f5bf2e745bbb6ff9fc84432345c)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:03.789225 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74d5e5-2191-4dde-a981-8d951891339e admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:03.847192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ef74d5e5-2191-4dde-a981-8d951891339e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43536f5bf2e745bbb6ff9fc84432345c'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:03.848034 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ef74d5e5-2191-4dde-a981-8d951891339e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43536f5bf2e745bbb6ff9fc84432345c'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:03.858505 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ef74d5e5-2191-4dde-a981-8d951891339e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43536f5bf2e745bbb6ff9fc84432345c'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:03.866505 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ef74d5e5-2191-4dde-a981-8d951891339e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43536f5bf2e745bbb6ff9fc84432345c'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:03.879107 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-ef74d5e5-2191-4dde-a981-8d951891339e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43536f5bf2e745bbb6ff9fc84432345c'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:03.885264 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1344/2695] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:18:03 2022] DELETE /identity/v3/users/43536f5bf2e745bbb6ff9fc84432345c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:03.890051 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d456740-cdf1-4022-9f55-08e4359b02aa None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:03.890612 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d456740-cdf1-4022-9f55-08e4359b02aa None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:03.890612 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d456740-cdf1-4022-9f55-08e4359b02aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:03.912060 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-7d456740-cdf1-4022-9f55-08e4359b02aa None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:03.913346 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7d456740-cdf1-4022-9f55-08e4359b02aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:03.964961 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:03.986685 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1352/2696] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:18:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:03.997729 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74d5e5-2191-4dde-a981-8d951891339e admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:04.000089 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ef74d5e5-2191-4dde-a981-8d951891339e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:04.020869 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:04.073038 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74d5e5-2191-4dde-a981-8d951891339e admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:04.077734 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96f4da-4854-48ab-a707-615a514e4df5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:04.078357 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96f4da-4854-48ab-a707-615a514e4df5 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:04.078652 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96f4da-4854-48ab-a707-615a514e4df5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:04.078896 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96f4da-4854-48ab-a707-615a514e4df5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:04.088979 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96f4da-4854-48ab-a707-615a514e4df5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:04.090449 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96f4da-4854-48ab-a707-615a514e4df5 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:04.100786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1345/2697] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:18:04.406408 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d456740-cdf1-4022-9f55-08e4359b02aa None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:04.408144 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7d456740-cdf1-4022-9f55-08e4359b02aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:04.423081 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d456740-cdf1-4022-9f55-08e4359b02aa None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:04.428663 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d83ce45-c85c-4e87-9e36-9af2e81ae2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:04.429345 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d83ce45-c85c-4e87-9e36-9af2e81ae2c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:04.429624 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d83ce45-c85c-4e87-9e36-9af2e81ae2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:04.429899 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d83ce45-c85c-4e87-9e36-9af2e81ae2c1 admin admin] PATH_INFO: `/v3/projects/9d0b425de0a84dfb9018c530ad0ec300` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:04.431078 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d83ce45-c85c-4e87-9e36-9af2e81ae2c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d0b425de0a84dfb9018c530ad0ec300)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:04.434114 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d83ce45-c85c-4e87-9e36-9af2e81ae2c1 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:04.492002 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2d83ce45-c85c-4e87-9e36-9af2e81ae2c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d0b425de0a84dfb9018c530ad0ec300', 'request_id': 'req-2d83ce45-c85c-4e87-9e36-9af2e81ae2c1'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:04.498287 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2d83ce45-c85c-4e87-9e36-9af2e81ae2c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d0b425de0a84dfb9018c530ad0ec300', 'request_id': 'req-2d83ce45-c85c-4e87-9e36-9af2e81ae2c1'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:04.506660 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2d83ce45-c85c-4e87-9e36-9af2e81ae2c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d0b425de0a84dfb9018c530ad0ec300', 'request_id': 'req-2d83ce45-c85c-4e87-9e36-9af2e81ae2c1'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:04.518751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2d83ce45-c85c-4e87-9e36-9af2e81ae2c1 admin admin] The token cache is being invalidate because project 9d0b425de0a84dfb9018c530ad0ec300 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:18:04.519037 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2d83ce45-c85c-4e87-9e36-9af2e81ae2c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:04.519869 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-2d83ce45-c85c-4e87-9e36-9af2e81ae2c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:04.521514 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1353/2698] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:18:04 2022] DELETE /identity/v3/projects/9d0b425de0a84dfb9018c530ad0ec300 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:04.526569 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96f4da-4854-48ab-a707-615a514e4df5 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:04.537677 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96f4da-4854-48ab-a707-615a514e4df5 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:04.601560 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e0d34-2b99-4241-9c1e-da161456385b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:04.602139 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e0d34-2b99-4241-9c1e-da161456385b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:04.602425 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e0d34-2b99-4241-9c1e-da161456385b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:04.602694 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e0d34-2b99-4241-9c1e-da161456385b admin admin] PATH_INFO: `/v3/users/b25b3b869ef4462394ab648cc0ef8b03` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:04.603925 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e0d34-2b99-4241-9c1e-da161456385b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b25b3b869ef4462394ab648cc0ef8b03)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:04.605911 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e0d34-2b99-4241-9c1e-da161456385b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:04.658323 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-fb9e0d34-2b99-4241-9c1e-da161456385b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b25b3b869ef4462394ab648cc0ef8b03'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:04.659014 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-fb9e0d34-2b99-4241-9c1e-da161456385b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b25b3b869ef4462394ab648cc0ef8b03'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:04.668420 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-fb9e0d34-2b99-4241-9c1e-da161456385b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b25b3b869ef4462394ab648cc0ef8b03'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:04.679796 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-fb9e0d34-2b99-4241-9c1e-da161456385b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b25b3b869ef4462394ab648cc0ef8b03'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:04.693588 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-fb9e0d34-2b99-4241-9c1e-da161456385b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b25b3b869ef4462394ab648cc0ef8b03'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:04.700074 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1346/2699] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:18:04 2022] DELETE /identity/v3/users/b25b3b869ef4462394ab648cc0ef8b03 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:05.033273 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d83ce45-c85c-4e87-9e36-9af2e81ae2c1 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:05.035550 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2d83ce45-c85c-4e87-9e36-9af2e81ae2c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:05.058487 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d83ce45-c85c-4e87-9e36-9af2e81ae2c1 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:05.121088 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f183e6-2a71-49e4-82e9-b10f762e0585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:05.121667 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f183e6-2a71-49e4-82e9-b10f762e0585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:05.121968 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f183e6-2a71-49e4-82e9-b10f762e0585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:05.122236 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f183e6-2a71-49e4-82e9-b10f762e0585 admin admin] PATH_INFO: `/v3/projects/22d2b657892d4beabc97a91d41d9041a` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:05.123435 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f183e6-2a71-49e4-82e9-b10f762e0585 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22d2b657892d4beabc97a91d41d9041a)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:05.125437 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f183e6-2a71-49e4-82e9-b10f762e0585 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:05.177859 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-07f183e6-2a71-49e4-82e9-b10f762e0585 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22d2b657892d4beabc97a91d41d9041a', 'request_id': 'req-07f183e6-2a71-49e4-82e9-b10f762e0585'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:05.183993 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-07f183e6-2a71-49e4-82e9-b10f762e0585 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22d2b657892d4beabc97a91d41d9041a', 'request_id': 'req-07f183e6-2a71-49e4-82e9-b10f762e0585'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:05.192898 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-07f183e6-2a71-49e4-82e9-b10f762e0585 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22d2b657892d4beabc97a91d41d9041a', 'request_id': 'req-07f183e6-2a71-49e4-82e9-b10f762e0585'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:05.201493 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:05.207141 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-07f183e6-2a71-49e4-82e9-b10f762e0585 admin admin] The token cache is being invalidate because project 22d2b657892d4beabc97a91d41d9041a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:18:05.207446 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-07f183e6-2a71-49e4-82e9-b10f762e0585 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:05.209107 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-07f183e6-2a71-49e4-82e9-b10f762e0585 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:05.211045 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1354/2700] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:18:05 2022] DELETE /identity/v3/projects/22d2b657892d4beabc97a91d41d9041a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:05.215936 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130327e2-5c49-43af-9414-addf04e790b7 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:05.216699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130327e2-5c49-43af-9414-addf04e790b7 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:05.216699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130327e2-5c49-43af-9414-addf04e790b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:05.238481 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-130327e2-5c49-43af-9414-addf04e790b7 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:05.239891 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-130327e2-5c49-43af-9414-addf04e790b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:05.273510 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:05.313669 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1347/2701] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:18:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:05.321759 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0cd68-c43e-4118-b42a-41c8db39ce8c None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:05.322056 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0cd68-c43e-4118-b42a-41c8db39ce8c None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:05.322320 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0cd68-c43e-4118-b42a-41c8db39ce8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:05.345396 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-3db0cd68-c43e-4118-b42a-41c8db39ce8c None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:05.346806 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3db0cd68-c43e-4118-b42a-41c8db39ce8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:05.355991 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1355/2702] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:18:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:05.360756 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130327e2-5c49-43af-9414-addf04e790b7 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:05.372171 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130327e2-5c49-43af-9414-addf04e790b7 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:05.378217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88253738-009f-4f81-8412-af0386456f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:05.378847 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88253738-009f-4f81-8412-af0386456f65 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:05.379095 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88253738-009f-4f81-8412-af0386456f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:05.379391 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88253738-009f-4f81-8412-af0386456f65 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:05.380092 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88253738-009f-4f81-8412-af0386456f65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:18:05.380462 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88253738-009f-4f81-8412-af0386456f65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:05.382133 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88253738-009f-4f81-8412-af0386456f65 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:05.397132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1348/2703] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:18:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:18:05.403495 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db0cd68-c43e-4118-b42a-41c8db39ce8c None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:05.417080 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db0cd68-c43e-4118-b42a-41c8db39ce8c None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:05.423032 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d6075-3f68-47a4-a6f1-d3ed5baa5eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:05.423702 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d6075-3f68-47a4-a6f1-d3ed5baa5eee admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:05.423959 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d6075-3f68-47a4-a6f1-d3ed5baa5eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:05.424221 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d6075-3f68-47a4-a6f1-d3ed5baa5eee admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:05.424940 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933d6075-3f68-47a4-a6f1-d3ed5baa5eee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:05.427021 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933d6075-3f68-47a4-a6f1-d3ed5baa5eee admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:05.463107 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1356/2704] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:18:05 2022] POST /identity/v3/projects => generated 387 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:18:05.467581 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88253738-009f-4f81-8412-af0386456f65 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:05.479922 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88253738-009f-4f81-8412-af0386456f65 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:05.565952 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc3562-eeaf-4b00-81cb-4859339f64d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:05.566833 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc3562-eeaf-4b00-81cb-4859339f64d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:05.567293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc3562-eeaf-4b00-81cb-4859339f64d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:05.567565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc3562-eeaf-4b00-81cb-4859339f64d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:05.568265 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc3562-eeaf-4b00-81cb-4859339f64d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:05.570358 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc3562-eeaf-4b00-81cb-4859339f64d3 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:05.594651 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1349/2705] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:18:05 2022] POST /identity/v3/users => generated 294 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:18:05.599544 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d6075-3f68-47a4-a6f1-d3ed5baa5eee admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:05.611219 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d6075-3f68-47a4-a6f1-d3ed5baa5eee admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:05.616976 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c6f38-d1af-4b7a-a3ab-11805d1c7a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:05.617762 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c6f38-d1af-4b7a-a3ab-11805d1c7a04 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:05.618179 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c6f38-d1af-4b7a-a3ab-11805d1c7a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:05.618498 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c6f38-d1af-4b7a-a3ab-11805d1c7a04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:05.619283 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c6f38-d1af-4b7a-a3ab-11805d1c7a04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:05.621052 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c6f38-d1af-4b7a-a3ab-11805d1c7a04 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:05.632856 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1357/2706] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:18:05.637365 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc3562-eeaf-4b00-81cb-4859339f64d3 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:05.648583 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc3562-eeaf-4b00-81cb-4859339f64d3 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:05.654587 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b39eb-74c7-431e-aa6e-328752d71456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:05.655228 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b39eb-74c7-431e-aa6e-328752d71456 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:05.655520 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b39eb-74c7-431e-aa6e-328752d71456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:05.655832 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b39eb-74c7-431e-aa6e-328752d71456 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:05.656558 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4b39eb-74c7-431e-aa6e-328752d71456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:05.658226 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4b39eb-74c7-431e-aa6e-328752d71456 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:05.671268 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1350/2707] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:18:05.676140 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c6f38-d1af-4b7a-a3ab-11805d1c7a04 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:05.695448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c6f38-d1af-4b7a-a3ab-11805d1c7a04 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:05.701034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1745bc-4d7a-4329-9ced-bdeff5cff24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:05.701735 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1745bc-4d7a-4329-9ced-bdeff5cff24e admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:05.702018 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1745bc-4d7a-4329-9ced-bdeff5cff24e admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:05.702294 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1745bc-4d7a-4329-9ced-bdeff5cff24e admin admin] PATH_INFO: `/v3/projects/1504bb71fbdd46caa288a9d5fe7984c4/users/07506882e2ff42f19e34d729d3a5c71f/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:05.720147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1745bc-4d7a-4329-9ced-bdeff5cff24e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1504bb71fbdd46caa288a9d5fe7984c4, user_id=07506882e2ff42f19e34d729d3a5c71f, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:05.723280 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1745bc-4d7a-4329-9ced-bdeff5cff24e admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:05.741090 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1358/2708] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:18:05 2022] PUT /identity/v3/projects/1504bb71fbdd46caa288a9d5fe7984c4/users/07506882e2ff42f19e34d729d3a5c71f/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:05.752204 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068283b-8e75-4c3f-9955-c76784be0493 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:05.752536 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068283b-8e75-4c3f-9955-c76784be0493 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:05.752814 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068283b-8e75-4c3f-9955-c76784be0493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:05.786832 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-6068283b-8e75-4c3f-9955-c76784be0493 None None] MFA Rules not processed for user `07506882e2ff42f19e34d729d3a5c71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:05.788250 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-6068283b-8e75-4c3f-9955-c76784be0493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:05.867730 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1351/2709] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:18:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:05.872965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1745bc-4d7a-4329-9ced-bdeff5cff24e admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:05.948491 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1745bc-4d7a-4329-9ced-bdeff5cff24e admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:05.955788 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9efd1d-8dba-451f-abfd-6df8ee200ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:05.956791 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9efd1d-8dba-451f-abfd-6df8ee200ab5 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:05.958147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9efd1d-8dba-451f-abfd-6df8ee200ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:05.958376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9efd1d-8dba-451f-abfd-6df8ee200ab5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:05.959514 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9efd1d-8dba-451f-abfd-6df8ee200ab5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:05.962624 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9efd1d-8dba-451f-abfd-6df8ee200ab5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:06.002886 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1359/2710] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:18:05 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:18:06.007879 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6068283b-8e75-4c3f-9955-c76784be0493 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:06.020658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6068283b-8e75-4c3f-9955-c76784be0493 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:06.056703 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:06.101142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b64de-f171-470e-afbc-43b6d3073514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:06.101947 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b64de-f171-470e-afbc-43b6d3073514 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:06.102172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b64de-f171-470e-afbc-43b6d3073514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:06.102536 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b64de-f171-470e-afbc-43b6d3073514 admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:06.103251 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b64de-f171-470e-afbc-43b6d3073514 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:06.105567 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b64de-f171-470e-afbc-43b6d3073514 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:06.128870 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1352/2711] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:18:06 2022] POST /identity/v3/users => generated 299 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:18:06.133164 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9efd1d-8dba-451f-abfd-6df8ee200ab5 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:06.144174 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9efd1d-8dba-451f-abfd-6df8ee200ab5 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:06.149505 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b687ef-ad1e-4b41-9fe5-1ef5f5fc3220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:06.150149 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b687ef-ad1e-4b41-9fe5-1ef5f5fc3220 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:06.150427 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b687ef-ad1e-4b41-9fe5-1ef5f5fc3220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:06.150699 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b687ef-ad1e-4b41-9fe5-1ef5f5fc3220 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:06.151400 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b687ef-ad1e-4b41-9fe5-1ef5f5fc3220 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:06.153054 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b687ef-ad1e-4b41-9fe5-1ef5f5fc3220 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:06.165005 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1360/2712] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:18:06.170371 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b64de-f171-470e-afbc-43b6d3073514 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:06.185799 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b64de-f171-470e-afbc-43b6d3073514 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:06.190884 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ed167-c4c3-42a1-9190-b04581b67b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:06.191652 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ed167-c4c3-42a1-9190-b04581b67b6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:06.192028 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ed167-c4c3-42a1-9190-b04581b67b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:06.192434 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ed167-c4c3-42a1-9190-b04581b67b6b admin admin] PATH_INFO: `/v3/projects/4cb23dff34c74f5899e04dcb2d87d7e2/users/d44682f73f1344dfbe42292463dd928b/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:06.209882 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935ed167-c4c3-42a1-9190-b04581b67b6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cb23dff34c74f5899e04dcb2d87d7e2, user_id=d44682f73f1344dfbe42292463dd928b, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:06.212822 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935ed167-c4c3-42a1-9190-b04581b67b6b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:06.230710 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1353/2713] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:18:06 2022] PUT /identity/v3/projects/4cb23dff34c74f5899e04dcb2d87d7e2/users/d44682f73f1344dfbe42292463dd928b/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:06.235158 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b687ef-ad1e-4b41-9fe5-1ef5f5fc3220 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:06.246444 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b687ef-ad1e-4b41-9fe5-1ef5f5fc3220 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:06.311308 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67394cc1-a85b-4f8e-a03c-1aa3eb8dc8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:06.311955 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67394cc1-a85b-4f8e-a03c-1aa3eb8dc8af admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:06.312239 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67394cc1-a85b-4f8e-a03c-1aa3eb8dc8af admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:06.312536 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67394cc1-a85b-4f8e-a03c-1aa3eb8dc8af admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:06.313241 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67394cc1-a85b-4f8e-a03c-1aa3eb8dc8af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:06.314872 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67394cc1-a85b-4f8e-a03c-1aa3eb8dc8af admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:06.326235 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1361/2714] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:18:06.330882 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ed167-c4c3-42a1-9190-b04581b67b6b admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:06.342809 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ed167-c4c3-42a1-9190-b04581b67b6b admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:06.350756 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d41c4b-a036-4c3a-8066-c715043c195f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:06.351490 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d41c4b-a036-4c3a-8066-c715043c195f admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:06.351745 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d41c4b-a036-4c3a-8066-c715043c195f admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:06.352018 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d41c4b-a036-4c3a-8066-c715043c195f admin admin] PATH_INFO: `/v3/projects/4cb23dff34c74f5899e04dcb2d87d7e2/users/d44682f73f1344dfbe42292463dd928b/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:06.354241 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d41c4b-a036-4c3a-8066-c715043c195f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cb23dff34c74f5899e04dcb2d87d7e2, user_id=d44682f73f1344dfbe42292463dd928b, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:06.357073 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d41c4b-a036-4c3a-8066-c715043c195f admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:06.370435 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1354/2715] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:18:06 2022] PUT /identity/v3/projects/4cb23dff34c74f5899e04dcb2d87d7e2/users/d44682f73f1344dfbe42292463dd928b/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:18:06.382965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32df57e-1312-4e68-919d-5d8ae5459896 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:06.383264 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32df57e-1312-4e68-919d-5d8ae5459896 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:06.383527 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32df57e-1312-4e68-919d-5d8ae5459896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:06.392539 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:06.419299 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d32df57e-1312-4e68-919d-5d8ae5459896 None None] MFA Rules not processed for user `d44682f73f1344dfbe42292463dd928b`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:06.427235 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d32df57e-1312-4e68-919d-5d8ae5459896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:06.497735 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1362/2716] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:18:06 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:06.509661 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d41c4b-a036-4c3a-8066-c715043c195f admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:06.511799 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-63d41c4b-a036-4c3a-8066-c715043c195f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:06.581871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d41c4b-a036-4c3a-8066-c715043c195f admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:06.586567 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2871fdc-ecaf-4273-a1a1-058b13f33a59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:06.587236 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2871fdc-ecaf-4273-a1a1-058b13f33a59 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:06.587520 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2871fdc-ecaf-4273-a1a1-058b13f33a59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:06.587793 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2871fdc-ecaf-4273-a1a1-058b13f33a59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:06.598720 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2871fdc-ecaf-4273-a1a1-058b13f33a59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:06.600204 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2871fdc-ecaf-4273-a1a1-058b13f33a59 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:06.614142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1355/2717] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:06 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:18:07.283397 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32df57e-1312-4e68-919d-5d8ae5459896 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:07.358913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32df57e-1312-4e68-919d-5d8ae5459896 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:07.363982 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae2921-ea19-453b-9f55-4fe34cdd2879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:07.364707 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae2921-ea19-453b-9f55-4fe34cdd2879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:07.364974 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae2921-ea19-453b-9f55-4fe34cdd2879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:07.365245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae2921-ea19-453b-9f55-4fe34cdd2879 admin admin] PATH_INFO: `/v3/users/07506882e2ff42f19e34d729d3a5c71f` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:07.366590 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aae2921-ea19-453b-9f55-4fe34cdd2879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07506882e2ff42f19e34d729d3a5c71f)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:07.368668 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aae2921-ea19-453b-9f55-4fe34cdd2879 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:07.435494 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7aae2921-ea19-453b-9f55-4fe34cdd2879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07506882e2ff42f19e34d729d3a5c71f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:07.436238 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7aae2921-ea19-453b-9f55-4fe34cdd2879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07506882e2ff42f19e34d729d3a5c71f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:07.452736 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7aae2921-ea19-453b-9f55-4fe34cdd2879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07506882e2ff42f19e34d729d3a5c71f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:07.460616 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7aae2921-ea19-453b-9f55-4fe34cdd2879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07506882e2ff42f19e34d729d3a5c71f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:07.461461 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-7aae2921-ea19-453b-9f55-4fe34cdd2879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07506882e2ff42f19e34d729d3a5c71f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:07.478872 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1363/2718] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:18:07 2022] DELETE /identity/v3/users/07506882e2ff42f19e34d729d3a5c71f => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:07.483779 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49978496-9532-4062-bdda-638111a7d9e7 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:07.484433 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49978496-9532-4062-bdda-638111a7d9e7 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:07.484679 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49978496-9532-4062-bdda-638111a7d9e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:07.508247 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-49978496-9532-4062-bdda-638111a7d9e7 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:07.509694 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-49978496-9532-4062-bdda-638111a7d9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:07.579944 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1356/2719] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:18:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:07.590554 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae2921-ea19-453b-9f55-4fe34cdd2879 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:07.592236 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-7aae2921-ea19-453b-9f55-4fe34cdd2879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:07.603891 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:07.660159 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae2921-ea19-453b-9f55-4fe34cdd2879 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:07.664237 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1d9a8-0a23-4a65-abf6-9f5fd082811c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:07.664914 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1d9a8-0a23-4a65-abf6-9f5fd082811c service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:07.665199 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1d9a8-0a23-4a65-abf6-9f5fd082811c service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:07.665477 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1d9a8-0a23-4a65-abf6-9f5fd082811c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:07.676324 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab1d9a8-0a23-4a65-abf6-9f5fd082811c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:07.677801 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab1d9a8-0a23-4a65-abf6-9f5fd082811c service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:07.689240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1364/2720] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:18:08.013828 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49978496-9532-4062-bdda-638111a7d9e7 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:08.015441 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-49978496-9532-4062-bdda-638111a7d9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:08.029432 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49978496-9532-4062-bdda-638111a7d9e7 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:08.035039 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24a741-2b29-40a7-9419-3e1548e91a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:08.035900 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24a741-2b29-40a7-9419-3e1548e91a30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:08.036217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24a741-2b29-40a7-9419-3e1548e91a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:08.036501 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24a741-2b29-40a7-9419-3e1548e91a30 admin admin] PATH_INFO: `/v3/projects/1504bb71fbdd46caa288a9d5fe7984c4` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:08.037903 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f24a741-2b29-40a7-9419-3e1548e91a30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1504bb71fbdd46caa288a9d5fe7984c4)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:08.039993 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f24a741-2b29-40a7-9419-3e1548e91a30 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:08.066106 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:08.099211 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1f24a741-2b29-40a7-9419-3e1548e91a30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1504bb71fbdd46caa288a9d5fe7984c4', 'request_id': 'req-1f24a741-2b29-40a7-9419-3e1548e91a30'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:08.106067 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1f24a741-2b29-40a7-9419-3e1548e91a30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1504bb71fbdd46caa288a9d5fe7984c4', 'request_id': 'req-1f24a741-2b29-40a7-9419-3e1548e91a30'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:08.115499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1f24a741-2b29-40a7-9419-3e1548e91a30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1504bb71fbdd46caa288a9d5fe7984c4', 'request_id': 'req-1f24a741-2b29-40a7-9419-3e1548e91a30'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:08.127644 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1f24a741-2b29-40a7-9419-3e1548e91a30 admin admin] The token cache is being invalidate because project 1504bb71fbdd46caa288a9d5fe7984c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:18:08.128187 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1f24a741-2b29-40a7-9419-3e1548e91a30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:08.128684 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-1f24a741-2b29-40a7-9419-3e1548e91a30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:08.130342 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1357/2721] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:18:08 2022] DELETE /identity/v3/projects/1504bb71fbdd46caa288a9d5fe7984c4 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:08.134638 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1d9a8-0a23-4a65-abf6-9f5fd082811c service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:08.145827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1d9a8-0a23-4a65-abf6-9f5fd082811c service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:08.214532 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb580e9d-be5d-479a-9150-a98113e38263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:08.215111 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb580e9d-be5d-479a-9150-a98113e38263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:08.215482 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb580e9d-be5d-479a-9150-a98113e38263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:08.215718 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb580e9d-be5d-479a-9150-a98113e38263 admin admin] PATH_INFO: `/v3/users/d44682f73f1344dfbe42292463dd928b` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:08.216953 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb580e9d-be5d-479a-9150-a98113e38263 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d44682f73f1344dfbe42292463dd928b)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:08.218924 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb580e9d-be5d-479a-9150-a98113e38263 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:08.278865 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-eb580e9d-be5d-479a-9150-a98113e38263 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd44682f73f1344dfbe42292463dd928b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:08.279523 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-eb580e9d-be5d-479a-9150-a98113e38263 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd44682f73f1344dfbe42292463dd928b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:08.301733 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-eb580e9d-be5d-479a-9150-a98113e38263 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd44682f73f1344dfbe42292463dd928b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:08.309476 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-eb580e9d-be5d-479a-9150-a98113e38263 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd44682f73f1344dfbe42292463dd928b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:08.310238 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-eb580e9d-be5d-479a-9150-a98113e38263 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd44682f73f1344dfbe42292463dd928b'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:08.325241 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1365/2722] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:18:08 2022] DELETE /identity/v3/users/d44682f73f1344dfbe42292463dd928b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:08.659709 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24a741-2b29-40a7-9419-3e1548e91a30 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:08.661524 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-1f24a741-2b29-40a7-9419-3e1548e91a30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:08.675604 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24a741-2b29-40a7-9419-3e1548e91a30 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:08.744403 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466dc727-bb93-4da5-80a1-dece1d39da75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:08.745021 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466dc727-bb93-4da5-80a1-dece1d39da75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:08.745301 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466dc727-bb93-4da5-80a1-dece1d39da75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:08.745575 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466dc727-bb93-4da5-80a1-dece1d39da75 admin admin] PATH_INFO: `/v3/projects/4cb23dff34c74f5899e04dcb2d87d7e2` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:08.746817 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466dc727-bb93-4da5-80a1-dece1d39da75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cb23dff34c74f5899e04dcb2d87d7e2)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:08.748808 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466dc727-bb93-4da5-80a1-dece1d39da75 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:08.806859 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-466dc727-bb93-4da5-80a1-dece1d39da75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cb23dff34c74f5899e04dcb2d87d7e2', 'request_id': 'req-466dc727-bb93-4da5-80a1-dece1d39da75'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:08.814399 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-466dc727-bb93-4da5-80a1-dece1d39da75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cb23dff34c74f5899e04dcb2d87d7e2', 'request_id': 'req-466dc727-bb93-4da5-80a1-dece1d39da75'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:08.823520 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-466dc727-bb93-4da5-80a1-dece1d39da75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cb23dff34c74f5899e04dcb2d87d7e2', 'request_id': 'req-466dc727-bb93-4da5-80a1-dece1d39da75'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:08.836570 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-466dc727-bb93-4da5-80a1-dece1d39da75 admin admin] The token cache is being invalidate because project 4cb23dff34c74f5899e04dcb2d87d7e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:18:08.836869 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-466dc727-bb93-4da5-80a1-dece1d39da75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:08.837724 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-466dc727-bb93-4da5-80a1-dece1d39da75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:08.839597 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1358/2723] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:18:08 2022] DELETE /identity/v3/projects/4cb23dff34c74f5899e04dcb2d87d7e2 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:08.845052 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbfcc8-8a83-486d-83e8-0367b348258b None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:08.845330 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbfcc8-8a83-486d-83e8-0367b348258b None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:08.845618 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbfcc8-8a83-486d-83e8-0367b348258b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:08.868792 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-17dbfcc8-8a83-486d-83e8-0367b348258b None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:08.870200 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-17dbfcc8-8a83-486d-83e8-0367b348258b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:08.898287 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:08.940710 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1366/2724] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:18:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:08.949096 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f77797-2e58-4297-80fa-9b016d71b861 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:08.949393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f77797-2e58-4297-80fa-9b016d71b861 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:08.949665 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f77797-2e58-4297-80fa-9b016d71b861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:08.971116 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-12f77797-2e58-4297-80fa-9b016d71b861 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:08.972405 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-12f77797-2e58-4297-80fa-9b016d71b861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:08.986588 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1359/2725] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:18:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:08.991279 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dbfcc8-8a83-486d-83e8-0367b348258b None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:09.002053 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dbfcc8-8a83-486d-83e8-0367b348258b None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:09.007846 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a30d613-6cdc-44af-b293-8419e54e4852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:09.008535 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30d613-6cdc-44af-b293-8419e54e4852 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:09.008900 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30d613-6cdc-44af-b293-8419e54e4852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:09.009086 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30d613-6cdc-44af-b293-8419e54e4852 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:09.009782 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a30d613-6cdc-44af-b293-8419e54e4852 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:18:09.010089 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a30d613-6cdc-44af-b293-8419e54e4852 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:09.011862 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a30d613-6cdc-44af-b293-8419e54e4852 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:09.024932 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1367/2726] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:18:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:18:09.029736 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f77797-2e58-4297-80fa-9b016d71b861 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:09.040353 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f77797-2e58-4297-80fa-9b016d71b861 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:09.046059 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac24c0-a349-4b6a-8e97-623637d3f3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:09.046702 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac24c0-a349-4b6a-8e97-623637d3f3da admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:09.046979 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac24c0-a349-4b6a-8e97-623637d3f3da admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:09.047244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac24c0-a349-4b6a-8e97-623637d3f3da admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:09.047938 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac24c0-a349-4b6a-8e97-623637d3f3da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:09.049995 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac24c0-a349-4b6a-8e97-623637d3f3da admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:09.080162 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1360/2727] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:18:09 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:18:09.085507 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a30d613-6cdc-44af-b293-8419e54e4852 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:09.096304 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a30d613-6cdc-44af-b293-8419e54e4852 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:09.159080 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f79742-a58b-4ae2-bd95-9b7baadd2759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:09.160071 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f79742-a58b-4ae2-bd95-9b7baadd2759 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:09.160378 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f79742-a58b-4ae2-bd95-9b7baadd2759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:09.160668 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f79742-a58b-4ae2-bd95-9b7baadd2759 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:09.161348 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f79742-a58b-4ae2-bd95-9b7baadd2759 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:09.163372 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f79742-a58b-4ae2-bd95-9b7baadd2759 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:09.198878 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1368/2728] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:18:09 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:18:09.200373 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac24c0-a349-4b6a-8e97-623637d3f3da admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:09.218251 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac24c0-a349-4b6a-8e97-623637d3f3da admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:09.223882 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c309073-c5bc-4ec1-8809-12743c6248f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:09.224594 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c309073-c5bc-4ec1-8809-12743c6248f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:09.224872 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c309073-c5bc-4ec1-8809-12743c6248f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:09.225142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c309073-c5bc-4ec1-8809-12743c6248f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:09.225867 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c309073-c5bc-4ec1-8809-12743c6248f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:09.227508 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c309073-c5bc-4ec1-8809-12743c6248f2 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:09.230450 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:09.240680 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1361/2729] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:18:09.246099 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f79742-a58b-4ae2-bd95-9b7baadd2759 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:09.261341 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f79742-a58b-4ae2-bd95-9b7baadd2759 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:09.266674 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b282c2a-c512-4341-8f28-a450d1ea0340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:09.267315 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b282c2a-c512-4341-8f28-a450d1ea0340 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:09.267593 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b282c2a-c512-4341-8f28-a450d1ea0340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:09.267863 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b282c2a-c512-4341-8f28-a450d1ea0340 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:09.268640 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b282c2a-c512-4341-8f28-a450d1ea0340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:09.270340 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b282c2a-c512-4341-8f28-a450d1ea0340 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:09.283561 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1369/2730] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:18:09.288970 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c309073-c5bc-4ec1-8809-12743c6248f2 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:09.310990 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c309073-c5bc-4ec1-8809-12743c6248f2 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:09.316797 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277c3e6-4057-4b05-a47b-6b2ba6d291b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:09.317495 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277c3e6-4057-4b05-a47b-6b2ba6d291b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:09.317768 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277c3e6-4057-4b05-a47b-6b2ba6d291b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:09.318037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277c3e6-4057-4b05-a47b-6b2ba6d291b8 admin admin] PATH_INFO: `/v3/projects/a7ea91f4369e436d846a629dd5fb7b72/users/1fc9d8b068084a0687750ddf28d91f48/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:09.336149 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4277c3e6-4057-4b05-a47b-6b2ba6d291b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7ea91f4369e436d846a629dd5fb7b72, user_id=1fc9d8b068084a0687750ddf28d91f48, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:09.338878 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4277c3e6-4057-4b05-a47b-6b2ba6d291b8 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:09.355983 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1362/2731] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:18:09 2022] PUT /identity/v3/projects/a7ea91f4369e436d846a629dd5fb7b72/users/1fc9d8b068084a0687750ddf28d91f48/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:09.366712 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a08f1-11be-45e5-8196-c45d65065e1f None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:09.367310 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a08f1-11be-45e5-8196-c45d65065e1f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:09.367310 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a08f1-11be-45e5-8196-c45d65065e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:09.400518 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-3c1a08f1-11be-45e5-8196-c45d65065e1f None None] MFA Rules not processed for user `1fc9d8b068084a0687750ddf28d91f48`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:09.401780 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3c1a08f1-11be-45e5-8196-c45d65065e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:09.476578 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1370/2732] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:18:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:09.481477 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277c3e6-4057-4b05-a47b-6b2ba6d291b8 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:09.565377 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277c3e6-4057-4b05-a47b-6b2ba6d291b8 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:09.571233 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253eea57-19b9-4532-9a85-12e3a7c05fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:09.571941 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253eea57-19b9-4532-9a85-12e3a7c05fc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:09.572215 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253eea57-19b9-4532-9a85-12e3a7c05fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:09.572515 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253eea57-19b9-4532-9a85-12e3a7c05fc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:09.573191 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253eea57-19b9-4532-9a85-12e3a7c05fc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:09.575136 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253eea57-19b9-4532-9a85-12e3a7c05fc5 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:09.610552 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1363/2733] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:18:09 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:18:09.614989 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a08f1-11be-45e5-8196-c45d65065e1f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:09.626618 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a08f1-11be-45e5-8196-c45d65065e1f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:09.658999 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:09.696583 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818e46e-8db3-44c3-aa66-db9ea9971584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:09.697215 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818e46e-8db3-44c3-aa66-db9ea9971584 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:09.697675 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818e46e-8db3-44c3-aa66-db9ea9971584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:09.697820 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818e46e-8db3-44c3-aa66-db9ea9971584 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:09.698445 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a818e46e-8db3-44c3-aa66-db9ea9971584 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:09.700434 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a818e46e-8db3-44c3-aa66-db9ea9971584 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:09.726686 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1371/2734] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:18:09 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:18:09.731701 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253eea57-19b9-4532-9a85-12e3a7c05fc5 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:09.743686 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253eea57-19b9-4532-9a85-12e3a7c05fc5 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:09.748681 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c668c-a123-4bf2-8f78-bbb0fae9f267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:09.749372 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c668c-a123-4bf2-8f78-bbb0fae9f267 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:09.749627 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c668c-a123-4bf2-8f78-bbb0fae9f267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:09.749898 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c668c-a123-4bf2-8f78-bbb0fae9f267 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:09.750595 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c668c-a123-4bf2-8f78-bbb0fae9f267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:09.752230 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c668c-a123-4bf2-8f78-bbb0fae9f267 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:09.763843 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1364/2735] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:18:09.768139 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818e46e-8db3-44c3-aa66-db9ea9971584 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:09.780410 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818e46e-8db3-44c3-aa66-db9ea9971584 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:09.787017 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c77b138-1bd6-4526-9547-99af29c881a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:09.787693 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77b138-1bd6-4526-9547-99af29c881a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:09.787970 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77b138-1bd6-4526-9547-99af29c881a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:09.788235 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77b138-1bd6-4526-9547-99af29c881a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:09.788961 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c77b138-1bd6-4526-9547-99af29c881a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:09.790777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c77b138-1bd6-4526-9547-99af29c881a7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:09.802831 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1372/2736] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:18:09.807587 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c668c-a123-4bf2-8f78-bbb0fae9f267 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:09.818892 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c668c-a123-4bf2-8f78-bbb0fae9f267 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:09.824351 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65518f6-c959-4c18-8da1-9246a3f81d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:09.825031 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65518f6-c959-4c18-8da1-9246a3f81d0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:09.825307 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65518f6-c959-4c18-8da1-9246a3f81d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:09.825573 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65518f6-c959-4c18-8da1-9246a3f81d0d admin admin] PATH_INFO: `/v3/projects/a1104596c158445bbd43d079ed6eac07/users/ebcf523f75ae4116a4d237c2cdf0f356/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:09.842120 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65518f6-c959-4c18-8da1-9246a3f81d0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1104596c158445bbd43d079ed6eac07, user_id=ebcf523f75ae4116a4d237c2cdf0f356, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:09.844819 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65518f6-c959-4c18-8da1-9246a3f81d0d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:09.860992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1365/2737] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:18:09 2022] PUT /identity/v3/projects/a1104596c158445bbd43d079ed6eac07/users/ebcf523f75ae4116a4d237c2cdf0f356/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:09.873104 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aef1b5-42ec-42bc-9e0d-6ee6a23672f9 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:09.873392 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aef1b5-42ec-42bc-9e0d-6ee6a23672f9 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:09.873677 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aef1b5-42ec-42bc-9e0d-6ee6a23672f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:09.911485 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b7aef1b5-42ec-42bc-9e0d-6ee6a23672f9 None None] MFA Rules not processed for user `ebcf523f75ae4116a4d237c2cdf0f356`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:09.912860 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b7aef1b5-42ec-42bc-9e0d-6ee6a23672f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:09.988998 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1373/2738] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:18:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:18:09.994824 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65518f6-c959-4c18-8da1-9246a3f81d0d admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:10.069644 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65518f6-c959-4c18-8da1-9246a3f81d0d admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:10.074597 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ec61c-f0fe-469a-b9d3-2500329fcb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:10.075707 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ec61c-f0fe-469a-b9d3-2500329fcb5c admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:10.075707 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ec61c-f0fe-469a-b9d3-2500329fcb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:10.075707 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ec61c-f0fe-469a-b9d3-2500329fcb5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:10.076454 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ec61c-f0fe-469a-b9d3-2500329fcb5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:10.078378 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ec61c-f0fe-469a-b9d3-2500329fcb5c admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:10.109901 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1366/2739] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:18:09 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:18:10.115992 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aef1b5-42ec-42bc-9e0d-6ee6a23672f9 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:10.127310 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aef1b5-42ec-42bc-9e0d-6ee6a23672f9 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:10.196893 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2272fa1-1e6f-4d90-8ca3-975bcbdf0cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:10.197498 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2272fa1-1e6f-4d90-8ca3-975bcbdf0cb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:10.197777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2272fa1-1e6f-4d90-8ca3-975bcbdf0cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:10.198061 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2272fa1-1e6f-4d90-8ca3-975bcbdf0cb5 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:10.198722 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2272fa1-1e6f-4d90-8ca3-975bcbdf0cb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:10.200731 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2272fa1-1e6f-4d90-8ca3-975bcbdf0cb5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:10.229658 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1374/2740] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:18:10 2022] POST /identity/v3/users => generated 299 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:18:10.231283 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ec61c-f0fe-469a-b9d3-2500329fcb5c admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:10.243614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ec61c-f0fe-469a-b9d3-2500329fcb5c admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:10.249770 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630908ad-99e4-41e1-90dc-1cfa8319f7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:10.250421 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630908ad-99e4-41e1-90dc-1cfa8319f7a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:10.250699 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630908ad-99e4-41e1-90dc-1cfa8319f7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:10.250967 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630908ad-99e4-41e1-90dc-1cfa8319f7a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:10.251670 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630908ad-99e4-41e1-90dc-1cfa8319f7a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:10.253320 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630908ad-99e4-41e1-90dc-1cfa8319f7a5 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:10.256135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:10.266373 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1367/2741] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:18:10.270901 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2272fa1-1e6f-4d90-8ca3-975bcbdf0cb5 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:10.282424 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2272fa1-1e6f-4d90-8ca3-975bcbdf0cb5 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:10.289519 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462fdb0e-5514-45c3-ad2d-cc88cdd59f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:10.290334 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fdb0e-5514-45c3-ad2d-cc88cdd59f38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:10.290656 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fdb0e-5514-45c3-ad2d-cc88cdd59f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:10.290915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fdb0e-5514-45c3-ad2d-cc88cdd59f38 admin admin] PATH_INFO: `/v3/projects/d3a408eae50f4017b4c348bb48104a00/users/03dfc94abe5c4e578de19ebda00d4c19/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:10.309849 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462fdb0e-5514-45c3-ad2d-cc88cdd59f38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3a408eae50f4017b4c348bb48104a00, user_id=03dfc94abe5c4e578de19ebda00d4c19, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:10.314376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462fdb0e-5514-45c3-ad2d-cc88cdd59f38 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:10.326083 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:10.333381 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1375/2742] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:18:10 2022] PUT /identity/v3/projects/d3a408eae50f4017b4c348bb48104a00/users/03dfc94abe5c4e578de19ebda00d4c19/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:10.338568 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630908ad-99e4-41e1-90dc-1cfa8319f7a5 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:10.349830 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630908ad-99e4-41e1-90dc-1cfa8319f7a5 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:10.417755 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0236e4b0-b9cb-43f0-a77e-454ed25d03ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:10.418606 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236e4b0-b9cb-43f0-a77e-454ed25d03ad admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:10.418732 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236e4b0-b9cb-43f0-a77e-454ed25d03ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:10.418926 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236e4b0-b9cb-43f0-a77e-454ed25d03ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:10.419725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0236e4b0-b9cb-43f0-a77e-454ed25d03ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:10.421273 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0236e4b0-b9cb-43f0-a77e-454ed25d03ad admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:10.431768 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1368/2743] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:18:10.436245 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462fdb0e-5514-45c3-ad2d-cc88cdd59f38 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:10.448052 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462fdb0e-5514-45c3-ad2d-cc88cdd59f38 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:10.454176 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604de6d5-0410-4875-9624-0bf1473cdeed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:10.454846 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604de6d5-0410-4875-9624-0bf1473cdeed admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:10.455121 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604de6d5-0410-4875-9624-0bf1473cdeed admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:10.455386 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604de6d5-0410-4875-9624-0bf1473cdeed admin admin] PATH_INFO: `/v3/projects/d3a408eae50f4017b4c348bb48104a00/users/03dfc94abe5c4e578de19ebda00d4c19/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:10.457629 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604de6d5-0410-4875-9624-0bf1473cdeed admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3a408eae50f4017b4c348bb48104a00, user_id=03dfc94abe5c4e578de19ebda00d4c19, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:10.460265 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604de6d5-0410-4875-9624-0bf1473cdeed admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:10.473035 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1376/2744] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:18:10 2022] PUT /identity/v3/projects/d3a408eae50f4017b4c348bb48104a00/users/03dfc94abe5c4e578de19ebda00d4c19/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:10.483838 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca8e1fe-bcf4-49c3-b145-9df651004c4d None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:10.484126 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca8e1fe-bcf4-49c3-b145-9df651004c4d None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:10.484438 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca8e1fe-bcf4-49c3-b145-9df651004c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:10.515958 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-3ca8e1fe-bcf4-49c3-b145-9df651004c4d None None] MFA Rules not processed for user `03dfc94abe5c4e578de19ebda00d4c19`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:10.517276 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3ca8e1fe-bcf4-49c3-b145-9df651004c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:10.607271 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1369/2745] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:18:10 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:10.619108 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604de6d5-0410-4875-9624-0bf1473cdeed admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:10.621318 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-604de6d5-0410-4875-9624-0bf1473cdeed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:10.689533 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604de6d5-0410-4875-9624-0bf1473cdeed admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:10.693448 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c96de65-9016-43f8-a34c-47c0155f787e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:10.694086 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c96de65-9016-43f8-a34c-47c0155f787e service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:10.694365 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c96de65-9016-43f8-a34c-47c0155f787e service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:10.694637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c96de65-9016-43f8-a34c-47c0155f787e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:10.762068 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c96de65-9016-43f8-a34c-47c0155f787e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:10.763524 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c96de65-9016-43f8-a34c-47c0155f787e service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:10.772945 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1377/2746] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:18:12.334707 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca8e1fe-bcf4-49c3-b145-9df651004c4d None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:12.336320 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3ca8e1fe-bcf4-49c3-b145-9df651004c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:12.352195 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca8e1fe-bcf4-49c3-b145-9df651004c4d None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:12.356912 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa3b36-239e-4cf3-823e-8463c303e21a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:12.357549 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa3b36-239e-4cf3-823e-8463c303e21a service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:12.357826 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa3b36-239e-4cf3-823e-8463c303e21a service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:12.358096 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa3b36-239e-4cf3-823e-8463c303e21a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:12.425551 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa3b36-239e-4cf3-823e-8463c303e21a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:12.427105 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa3b36-239e-4cf3-823e-8463c303e21a service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:12.437518 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1370/2747] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:18:13.034563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c96de65-9016-43f8-a34c-47c0155f787e service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:13.047083 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c96de65-9016-43f8-a34c-47c0155f787e service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:13.051594 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6da504-0663-4655-b89e-f4940d5246f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:13.052246 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6da504-0663-4655-b89e-f4940d5246f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:13.052547 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6da504-0663-4655-b89e-f4940d5246f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:13.052819 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6da504-0663-4655-b89e-f4940d5246f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:13.063042 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6da504-0663-4655-b89e-f4940d5246f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:13.064549 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6da504-0663-4655-b89e-f4940d5246f9 service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:13.078397 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1378/2748] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:13 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:18:17.073201 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa3b36-239e-4cf3-823e-8463c303e21a service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:17.101985 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=100 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:17.156932 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa3b36-239e-4cf3-823e-8463c303e21a service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:17.161489 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d638c5-cbd0-41ed-ba72-fa57b4499a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:17.162155 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d638c5-cbd0-41ed-ba72-fa57b4499a8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:17.162429 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d638c5-cbd0-41ed-ba72-fa57b4499a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:17.162701 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d638c5-cbd0-41ed-ba72-fa57b4499a8b admin admin] PATH_INFO: `/v3/users/1fc9d8b068084a0687750ddf28d91f48` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:17.164020 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d638c5-cbd0-41ed-ba72-fa57b4499a8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fc9d8b068084a0687750ddf28d91f48)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:17.166061 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d638c5-cbd0-41ed-ba72-fa57b4499a8b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:17.217737 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-05d638c5-cbd0-41ed-ba72-fa57b4499a8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fc9d8b068084a0687750ddf28d91f48'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:17.218504 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-05d638c5-cbd0-41ed-ba72-fa57b4499a8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fc9d8b068084a0687750ddf28d91f48'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:17.228595 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-05d638c5-cbd0-41ed-ba72-fa57b4499a8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fc9d8b068084a0687750ddf28d91f48'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:17.236160 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-05d638c5-cbd0-41ed-ba72-fa57b4499a8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fc9d8b068084a0687750ddf28d91f48'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:17.249313 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-05d638c5-cbd0-41ed-ba72-fa57b4499a8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fc9d8b068084a0687750ddf28d91f48'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:17.256767 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1371/2749] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:18:17 2022] DELETE /identity/v3/users/1fc9d8b068084a0687750ddf28d91f48 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:17.265780 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e3181-dc76-4751-9e90-f938d0ba385d None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:17.266323 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e3181-dc76-4751-9e90-f938d0ba385d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:17.266595 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e3181-dc76-4751-9e90-f938d0ba385d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:17.288079 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-fa3e3181-dc76-4751-9e90-f938d0ba385d None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:17.289350 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fa3e3181-dc76-4751-9e90-f938d0ba385d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:17.356556 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1379/2750] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:18:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:17.367057 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d638c5-cbd0-41ed-ba72-fa57b4499a8b admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:17.368572 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-05d638c5-cbd0-41ed-ba72-fa57b4499a8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:17.430559 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d638c5-cbd0-41ed-ba72-fa57b4499a8b admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:17.435097 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07acfd81-2d23-43da-aacf-b7529945d644 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:17.435748 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07acfd81-2d23-43da-aacf-b7529945d644 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:17.436028 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07acfd81-2d23-43da-aacf-b7529945d644 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:17.436304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07acfd81-2d23-43da-aacf-b7529945d644 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:17.447279 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07acfd81-2d23-43da-aacf-b7529945d644 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:17.448800 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07acfd81-2d23-43da-aacf-b7529945d644 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:17.459669 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1372/2751] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:18:17.743950 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e3181-dc76-4751-9e90-f938d0ba385d None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:17.746865 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-fa3e3181-dc76-4751-9e90-f938d0ba385d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:17.769858 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e3181-dc76-4751-9e90-f938d0ba385d None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:17.776873 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55490641-6424-47a4-b6a5-92badb5cdbd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:17.777664 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55490641-6424-47a4-b6a5-92badb5cdbd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:17.777985 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55490641-6424-47a4-b6a5-92badb5cdbd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:17.778298 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55490641-6424-47a4-b6a5-92badb5cdbd4 admin admin] PATH_INFO: `/v3/projects/a7ea91f4369e436d846a629dd5fb7b72` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:17.779566 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55490641-6424-47a4-b6a5-92badb5cdbd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7ea91f4369e436d846a629dd5fb7b72)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:17.781633 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55490641-6424-47a4-b6a5-92badb5cdbd4 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:17.820504 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:17.856637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-55490641-6424-47a4-b6a5-92badb5cdbd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7ea91f4369e436d846a629dd5fb7b72', 'request_id': 'req-55490641-6424-47a4-b6a5-92badb5cdbd4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:17.863126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-55490641-6424-47a4-b6a5-92badb5cdbd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7ea91f4369e436d846a629dd5fb7b72', 'request_id': 'req-55490641-6424-47a4-b6a5-92badb5cdbd4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:17.872038 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-55490641-6424-47a4-b6a5-92badb5cdbd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7ea91f4369e436d846a629dd5fb7b72', 'request_id': 'req-55490641-6424-47a4-b6a5-92badb5cdbd4'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:17.885486 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-55490641-6424-47a4-b6a5-92badb5cdbd4 admin admin] The token cache is being invalidate because project a7ea91f4369e436d846a629dd5fb7b72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:18:17.885743 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-55490641-6424-47a4-b6a5-92badb5cdbd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:17.886566 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-55490641-6424-47a4-b6a5-92badb5cdbd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:17.888309 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1380/2752] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:18:17 2022] DELETE /identity/v3/projects/a7ea91f4369e436d846a629dd5fb7b72 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:17.892794 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07acfd81-2d23-43da-aacf-b7529945d644 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:17.903274 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07acfd81-2d23-43da-aacf-b7529945d644 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:17.946592 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:17.971328 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fccc0f-59ae-45f2-b9f5-a48a67810730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:17.971992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fccc0f-59ae-45f2-b9f5-a48a67810730 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:17.972288 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fccc0f-59ae-45f2-b9f5-a48a67810730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:17.972574 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fccc0f-59ae-45f2-b9f5-a48a67810730 admin admin] PATH_INFO: `/v3/users/ebcf523f75ae4116a4d237c2cdf0f356` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:17.973830 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fccc0f-59ae-45f2-b9f5-a48a67810730 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebcf523f75ae4116a4d237c2cdf0f356)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:17.975773 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fccc0f-59ae-45f2-b9f5-a48a67810730 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:18.025467 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-31fccc0f-59ae-45f2-b9f5-a48a67810730 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebcf523f75ae4116a4d237c2cdf0f356'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:18.026163 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-31fccc0f-59ae-45f2-b9f5-a48a67810730 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebcf523f75ae4116a4d237c2cdf0f356'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:18.035660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-31fccc0f-59ae-45f2-b9f5-a48a67810730 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebcf523f75ae4116a4d237c2cdf0f356'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:18.043135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-31fccc0f-59ae-45f2-b9f5-a48a67810730 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebcf523f75ae4116a4d237c2cdf0f356'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:18.056013 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-31fccc0f-59ae-45f2-b9f5-a48a67810730 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebcf523f75ae4116a4d237c2cdf0f356'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:18.063132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1373/2753] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:18:17 2022] DELETE /identity/v3/users/ebcf523f75ae4116a4d237c2cdf0f356 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:18.372512 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55490641-6424-47a4-b6a5-92badb5cdbd4 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:18.375453 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-55490641-6424-47a4-b6a5-92badb5cdbd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:18.392548 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55490641-6424-47a4-b6a5-92badb5cdbd4 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:18.454670 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a1c664-65f9-4485-b590-4c503adca7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:18.455972 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1c664-65f9-4485-b590-4c503adca7dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:18.456259 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1c664-65f9-4485-b590-4c503adca7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:18.456568 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1c664-65f9-4485-b590-4c503adca7dc admin admin] PATH_INFO: `/v3/projects/a1104596c158445bbd43d079ed6eac07` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:18.458398 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a1c664-65f9-4485-b590-4c503adca7dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1104596c158445bbd43d079ed6eac07)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:18.460965 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a1c664-65f9-4485-b590-4c503adca7dc admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:18.516637 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d4a1c664-65f9-4485-b590-4c503adca7dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1104596c158445bbd43d079ed6eac07', 'request_id': 'req-d4a1c664-65f9-4485-b590-4c503adca7dc'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:18.522400 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d4a1c664-65f9-4485-b590-4c503adca7dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1104596c158445bbd43d079ed6eac07', 'request_id': 'req-d4a1c664-65f9-4485-b590-4c503adca7dc'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:18.530942 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d4a1c664-65f9-4485-b590-4c503adca7dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1104596c158445bbd43d079ed6eac07', 'request_id': 'req-d4a1c664-65f9-4485-b590-4c503adca7dc'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:18.543628 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d4a1c664-65f9-4485-b590-4c503adca7dc admin admin] The token cache is being invalidate because project a1104596c158445bbd43d079ed6eac07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:18:18.543918 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d4a1c664-65f9-4485-b590-4c503adca7dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:18.544588 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-d4a1c664-65f9-4485-b590-4c503adca7dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:18.546487 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1381/2754] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:18:18 2022] DELETE /identity/v3/projects/a1104596c158445bbd43d079ed6eac07 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:18.550565 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fccc0f-59ae-45f2-b9f5-a48a67810730 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:18.552587 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-31fccc0f-59ae-45f2-b9f5-a48a67810730 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:18.567027 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fccc0f-59ae-45f2-b9f5-a48a67810730 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:18.629794 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df5456-1731-45f7-98c4-531a67ade331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:18.630358 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df5456-1731-45f7-98c4-531a67ade331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:18.630636 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df5456-1731-45f7-98c4-531a67ade331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:18.630912 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df5456-1731-45f7-98c4-531a67ade331 admin admin] PATH_INFO: `/v3/users/03dfc94abe5c4e578de19ebda00d4c19` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:18.632096 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df5456-1731-45f7-98c4-531a67ade331 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03dfc94abe5c4e578de19ebda00d4c19)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:18.634230 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df5456-1731-45f7-98c4-531a67ade331 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:18.684589 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-c7df5456-1731-45f7-98c4-531a67ade331 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03dfc94abe5c4e578de19ebda00d4c19'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:18.685328 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-c7df5456-1731-45f7-98c4-531a67ade331 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03dfc94abe5c4e578de19ebda00d4c19'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:18.694387 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-c7df5456-1731-45f7-98c4-531a67ade331 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03dfc94abe5c4e578de19ebda00d4c19'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:18.700501 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:18.704080 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-c7df5456-1731-45f7-98c4-531a67ade331 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03dfc94abe5c4e578de19ebda00d4c19'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:18.718010 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-c7df5456-1731-45f7-98c4-531a67ade331 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03dfc94abe5c4e578de19ebda00d4c19'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:18.724496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1374/2755] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:18:18 2022] DELETE /identity/v3/users/03dfc94abe5c4e578de19ebda00d4c19 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:19.035151 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a1c664-65f9-4485-b590-4c503adca7dc admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:19.037324 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d4a1c664-65f9-4485-b590-4c503adca7dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:19.055837 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a1c664-65f9-4485-b590-4c503adca7dc admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:19.099830 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:19.125236 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89f844-0706-4d45-8761-004078d7945f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:19.126169 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89f844-0706-4d45-8761-004078d7945f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:19.126607 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89f844-0706-4d45-8761-004078d7945f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:19.127029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89f844-0706-4d45-8761-004078d7945f admin admin] PATH_INFO: `/v3/projects/d3a408eae50f4017b4c348bb48104a00` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:19.129033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89f844-0706-4d45-8761-004078d7945f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3a408eae50f4017b4c348bb48104a00)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:19.131138 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89f844-0706-4d45-8761-004078d7945f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:19.191017 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5d89f844-0706-4d45-8761-004078d7945f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3a408eae50f4017b4c348bb48104a00', 'request_id': 'req-5d89f844-0706-4d45-8761-004078d7945f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:19.196851 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5d89f844-0706-4d45-8761-004078d7945f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3a408eae50f4017b4c348bb48104a00', 'request_id': 'req-5d89f844-0706-4d45-8761-004078d7945f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:19.206329 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5d89f844-0706-4d45-8761-004078d7945f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3a408eae50f4017b4c348bb48104a00', 'request_id': 'req-5d89f844-0706-4d45-8761-004078d7945f'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:19.218068 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5d89f844-0706-4d45-8761-004078d7945f admin admin] The token cache is being invalidate because project d3a408eae50f4017b4c348bb48104a00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:18:19.218348 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5d89f844-0706-4d45-8761-004078d7945f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:19.219023 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-5d89f844-0706-4d45-8761-004078d7945f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:19.220624 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1382/2756] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:18:19 2022] DELETE /identity/v3/projects/d3a408eae50f4017b4c348bb48104a00 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:27.885030 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df5456-1731-45f7-98c4-531a67ade331 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:27.886606 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c7df5456-1731-45f7-98c4-531a67ade331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:27.950739 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df5456-1731-45f7-98c4-531a67ade331 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:28.013536 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff104d-a836-4c0b-801e-b6411f8b1051 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:28.014187 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff104d-a836-4c0b-801e-b6411f8b1051 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:28.014448 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff104d-a836-4c0b-801e-b6411f8b1051 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:28.014759 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff104d-a836-4c0b-801e-b6411f8b1051 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:28.016904 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-4fff104d-a836-4c0b-801e-b6411f8b1051 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:28.119267 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff104d-a836-4c0b-801e-b6411f8b1051 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:28.121347 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff104d-a836-4c0b-801e-b6411f8b1051 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:28.150246 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1375/2757] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:18:29.214989 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=32,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:35.072433 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89f844-0706-4d45-8761-004078d7945f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:35.074230 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-5d89f844-0706-4d45-8761-004078d7945f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:35.089201 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89f844-0706-4d45-8761-004078d7945f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:35.093183 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db261cea-f711-4c50-99c3-12ffab53b0ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:35.093823 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db261cea-f711-4c50-99c3-12ffab53b0ff service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:35.094107 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db261cea-f711-4c50-99c3-12ffab53b0ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:35.094377 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db261cea-f711-4c50-99c3-12ffab53b0ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:35.096314 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-db261cea-f711-4c50-99c3-12ffab53b0ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:35.111725 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db261cea-f711-4c50-99c3-12ffab53b0ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:35.113240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db261cea-f711-4c50-99c3-12ffab53b0ff service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:35.126197 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1383/2758] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:18:38.147288 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=94,keystone:DELETE=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:42.604886 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3e0ff-61b6-463b-a353-ad924743b096 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:42.605637 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3e0ff-61b6-463b-a353-ad924743b096 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:42.605756 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3e0ff-61b6-463b-a353-ad924743b096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:42.627567 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-0fd3e0ff-61b6-463b-a353-ad924743b096 None None] MFA Rules not processed for user `a9fc33bdca304c33b6a78122a54f492a`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:42.628932 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0fd3e0ff-61b6-463b-a353-ad924743b096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:42.734071 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1376/2759] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:18:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:45.120213 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=6 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:51.794430 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db261cea-f711-4c50-99c3-12ffab53b0ff service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:51.796055 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-db261cea-f711-4c50-99c3-12ffab53b0ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:51.867358 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db261cea-f711-4c50-99c3-12ffab53b0ff service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:51.871091 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88211658-6889-48fb-9897-64cdd3cfbf97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:51.871768 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88211658-6889-48fb-9897-64cdd3cfbf97 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:51.872038 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88211658-6889-48fb-9897-64cdd3cfbf97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:51.872388 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88211658-6889-48fb-9897-64cdd3cfbf97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:51.874170 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-88211658-6889-48fb-9897-64cdd3cfbf97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:51.891349 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88211658-6889-48fb-9897-64cdd3cfbf97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:51.893078 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88211658-6889-48fb-9897-64cdd3cfbf97 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:51.904080 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1384/2760] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:18:52.720398 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=25 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:19:01.899292 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=26 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:19:16.998841 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3e0ff-61b6-463b-a353-ad924743b096 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:17.000496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0fd3e0ff-61b6-463b-a353-ad924743b096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:19:17.023680 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3e0ff-61b6-463b-a353-ad924743b096 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:17.030439 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d79ace-5c86-4efc-aa2b-e09475f17b0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:17.031093 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d79ace-5c86-4efc-aa2b-e09475f17b0a service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:17.031372 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d79ace-5c86-4efc-aa2b-e09475f17b0a service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:17.031652 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d79ace-5c86-4efc-aa2b-e09475f17b0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:17.034277 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-00d79ace-5c86-4efc-aa2b-e09475f17b0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:19:17.049895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d79ace-5c86-4efc-aa2b-e09475f17b0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:17.051521 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d79ace-5c86-4efc-aa2b-e09475f17b0a service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:17.064865 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1377/2761] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:19:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:19:23.922423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88211658-6889-48fb-9897-64cdd3cfbf97 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:23.941781 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88211658-6889-48fb-9897-64cdd3cfbf97 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:23.950713 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e7526-9d2e-41ee-bf1f-d231ea9beaf7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:23.950713 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e7526-9d2e-41ee-bf1f-d231ea9beaf7 service nova] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:23.950713 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e7526-9d2e-41ee-bf1f-d231ea9beaf7 service nova] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:23.950713 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e7526-9d2e-41ee-bf1f-d231ea9beaf7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:23.953409 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d11e7526-9d2e-41ee-bf1f-d231ea9beaf7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:19:23.969213 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e7526-9d2e-41ee-bf1f-d231ea9beaf7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:23.971076 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e7526-9d2e-41ee-bf1f-d231ea9beaf7 service nova] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:23.985583 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1385/2762] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:19:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:19:27.059312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=6 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:19:29.713253 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d79ace-5c86-4efc-aa2b-e09475f17b0a service nova] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:29.723485 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d79ace-5c86-4efc-aa2b-e09475f17b0a service nova] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:29.727554 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7eba3-2bf1-4d1e-8159-8c99936f56e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:29.728200 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7eba3-2bf1-4d1e-8159-8c99936f56e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:29.728730 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7eba3-2bf1-4d1e-8159-8c99936f56e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:29.729108 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7eba3-2bf1-4d1e-8159-8c99936f56e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:29.730931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c9a7eba3-2bf1-4d1e-8159-8c99936f56e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:19:29.801780 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7eba3-2bf1-4d1e-8159-8c99936f56e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:29.803257 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7eba3-2bf1-4d1e-8159-8c99936f56e9 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:29.815299 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1378/2763] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:19:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:19:33.978897 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=6 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:19:37.293228 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e7526-9d2e-41ee-bf1f-d231ea9beaf7 service nova] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:37.294748 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d11e7526-9d2e-41ee-bf1f-d231ea9beaf7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:19:37.309312 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e7526-9d2e-41ee-bf1f-d231ea9beaf7 service nova] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:37.314593 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21dd8ac-75b2-4247-8a11-f9047798e3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:37.315242 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dd8ac-75b2-4247-8a11-f9047798e3ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:37.315523 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dd8ac-75b2-4247-8a11-f9047798e3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:37.316720 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dd8ac-75b2-4247-8a11-f9047798e3ff admin admin] PATH_INFO: `/v3/users/8e30910692b4474092f1585a671e2f41` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:37.318073 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21dd8ac-75b2-4247-8a11-f9047798e3ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e30910692b4474092f1585a671e2f41)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:37.320136 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21dd8ac-75b2-4247-8a11-f9047798e3ff admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:37.375890 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b21dd8ac-75b2-4247-8a11-f9047798e3ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e30910692b4474092f1585a671e2f41'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:37.376567 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b21dd8ac-75b2-4247-8a11-f9047798e3ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e30910692b4474092f1585a671e2f41'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:37.389711 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b21dd8ac-75b2-4247-8a11-f9047798e3ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e30910692b4474092f1585a671e2f41'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:37.396711 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b21dd8ac-75b2-4247-8a11-f9047798e3ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e30910692b4474092f1585a671e2f41'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:37.397426 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-b21dd8ac-75b2-4247-8a11-f9047798e3ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e30910692b4474092f1585a671e2f41'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:37.412829 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1386/2764] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:19:37 2022] DELETE /identity/v3/users/8e30910692b4474092f1585a671e2f41 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:19:37.616515 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7eba3-2bf1-4d1e-8159-8c99936f56e9 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:37.632409 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7eba3-2bf1-4d1e-8159-8c99936f56e9 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:37.697121 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e931a499-70e4-4526-9548-4456fa201dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:37.697881 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931a499-70e4-4526-9548-4456fa201dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:37.698135 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931a499-70e4-4526-9548-4456fa201dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:37.698396 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931a499-70e4-4526-9548-4456fa201dd5 admin admin] PATH_INFO: `/v3/projects/f0a6c07d69ef4cf4a545f48e7079044d` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:37.699567 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e931a499-70e4-4526-9548-4456fa201dd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0a6c07d69ef4cf4a545f48e7079044d)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:37.701516 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e931a499-70e4-4526-9548-4456fa201dd5 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:37.760259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e931a499-70e4-4526-9548-4456fa201dd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0a6c07d69ef4cf4a545f48e7079044d', 'request_id': 'req-e931a499-70e4-4526-9548-4456fa201dd5'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:37.766756 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e931a499-70e4-4526-9548-4456fa201dd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0a6c07d69ef4cf4a545f48e7079044d', 'request_id': 'req-e931a499-70e4-4526-9548-4456fa201dd5'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:37.775177 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e931a499-70e4-4526-9548-4456fa201dd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0a6c07d69ef4cf4a545f48e7079044d', 'request_id': 'req-e931a499-70e4-4526-9548-4456fa201dd5'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:37.786174 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e931a499-70e4-4526-9548-4456fa201dd5 admin admin] The token cache is being invalidate because project f0a6c07d69ef4cf4a545f48e7079044d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:19:37.786432 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e931a499-70e4-4526-9548-4456fa201dd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:37.787074 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-e931a499-70e4-4526-9548-4456fa201dd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:37.788672 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1379/2765] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:19:37 2022] DELETE /identity/v3/projects/f0a6c07d69ef4cf4a545f48e7079044d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:19:37.792831 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21dd8ac-75b2-4247-8a11-f9047798e3ff admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:37.794286 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b21dd8ac-75b2-4247-8a11-f9047798e3ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:19:37.807971 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21dd8ac-75b2-4247-8a11-f9047798e3ff admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:37.871188 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a75a4-103e-486f-9e46-a7819fa46b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:37.871844 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a75a4-103e-486f-9e46-a7819fa46b49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:37.872130 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a75a4-103e-486f-9e46-a7819fa46b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:37.872437 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a75a4-103e-486f-9e46-a7819fa46b49 admin admin] PATH_INFO: `/v3/users/612318edeb824e3abd82a00fc7fdc8df` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:37.873692 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a75a4-103e-486f-9e46-a7819fa46b49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=612318edeb824e3abd82a00fc7fdc8df)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:37.875633 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a75a4-103e-486f-9e46-a7819fa46b49 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:37.935450 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e40a75a4-103e-486f-9e46-a7819fa46b49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '612318edeb824e3abd82a00fc7fdc8df'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:37.936125 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e40a75a4-103e-486f-9e46-a7819fa46b49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '612318edeb824e3abd82a00fc7fdc8df'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:37.950450 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e40a75a4-103e-486f-9e46-a7819fa46b49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '612318edeb824e3abd82a00fc7fdc8df'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:37.959452 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e40a75a4-103e-486f-9e46-a7819fa46b49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '612318edeb824e3abd82a00fc7fdc8df'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:37.960293 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e40a75a4-103e-486f-9e46-a7819fa46b49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '612318edeb824e3abd82a00fc7fdc8df'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:37.962404 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:19:37.983023 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1387/2766] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:19:37 2022] DELETE /identity/v3/users/612318edeb824e3abd82a00fc7fdc8df => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:19:38.200148 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e931a499-70e4-4526-9548-4456fa201dd5 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:38.202597 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e931a499-70e4-4526-9548-4456fa201dd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:19:38.222916 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e931a499-70e4-4526-9548-4456fa201dd5 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:38.256132 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:19:38.313385 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661675d3-1c54-48ad-93e4-8b3933962112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:38.313966 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661675d3-1c54-48ad-93e4-8b3933962112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:38.314251 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661675d3-1c54-48ad-93e4-8b3933962112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:38.314548 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661675d3-1c54-48ad-93e4-8b3933962112 admin admin] PATH_INFO: `/v3/projects/fb0baf0e988143afb181a1bf178a1c53` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:38.315758 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661675d3-1c54-48ad-93e4-8b3933962112 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb0baf0e988143afb181a1bf178a1c53)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:38.317801 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661675d3-1c54-48ad-93e4-8b3933962112 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:38.368664 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-661675d3-1c54-48ad-93e4-8b3933962112 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb0baf0e988143afb181a1bf178a1c53', 'request_id': 'req-661675d3-1c54-48ad-93e4-8b3933962112'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:38.379112 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-661675d3-1c54-48ad-93e4-8b3933962112 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb0baf0e988143afb181a1bf178a1c53', 'request_id': 'req-661675d3-1c54-48ad-93e4-8b3933962112'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:38.389640 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-661675d3-1c54-48ad-93e4-8b3933962112 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb0baf0e988143afb181a1bf178a1c53', 'request_id': 'req-661675d3-1c54-48ad-93e4-8b3933962112'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:38.405576 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-661675d3-1c54-48ad-93e4-8b3933962112 admin admin] The token cache is being invalidate because project fb0baf0e988143afb181a1bf178a1c53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:19:38.405854 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-661675d3-1c54-48ad-93e4-8b3933962112 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:38.406638 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-661675d3-1c54-48ad-93e4-8b3933962112 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:38.408672 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1380/2767] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:19:38 2022] DELETE /identity/v3/projects/fb0baf0e988143afb181a1bf178a1c53 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:19:47.978642 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:19:48.401656 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:19:53.034421 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a75a4-103e-486f-9e46-a7819fa46b49 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:53.037073 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e40a75a4-103e-486f-9e46-a7819fa46b49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:19:53.060051 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a75a4-103e-486f-9e46-a7819fa46b49 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:53.152450 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc876e0-9a8b-44fb-a75f-87df1843a075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:53.153772 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc876e0-9a8b-44fb-a75f-87df1843a075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:53.154050 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc876e0-9a8b-44fb-a75f-87df1843a075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:53.154290 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc876e0-9a8b-44fb-a75f-87df1843a075 admin admin] PATH_INFO: `/v3/users/a9fc33bdca304c33b6a78122a54f492a` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:53.155615 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc876e0-9a8b-44fb-a75f-87df1843a075 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9fc33bdca304c33b6a78122a54f492a)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:53.157980 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc876e0-9a8b-44fb-a75f-87df1843a075 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:53.224681 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-0dc876e0-9a8b-44fb-a75f-87df1843a075 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9fc33bdca304c33b6a78122a54f492a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:53.225449 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-0dc876e0-9a8b-44fb-a75f-87df1843a075 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9fc33bdca304c33b6a78122a54f492a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:53.241783 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-0dc876e0-9a8b-44fb-a75f-87df1843a075 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9fc33bdca304c33b6a78122a54f492a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:53.250396 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-0dc876e0-9a8b-44fb-a75f-87df1843a075 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9fc33bdca304c33b6a78122a54f492a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:53.251273 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-0dc876e0-9a8b-44fb-a75f-87df1843a075 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9fc33bdca304c33b6a78122a54f492a'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:53.269037 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1388/2768] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:19:53 2022] DELETE /identity/v3/users/a9fc33bdca304c33b6a78122a54f492a => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:19:53.469580 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661675d3-1c54-48ad-93e4-8b3933962112 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:53.471086 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-661675d3-1c54-48ad-93e4-8b3933962112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:19:53.485328 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661675d3-1c54-48ad-93e4-8b3933962112 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:53.551492 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54734f35-1bb7-40ee-849b-6cc7e62a303a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:53.552153 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54734f35-1bb7-40ee-849b-6cc7e62a303a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:53.552481 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54734f35-1bb7-40ee-849b-6cc7e62a303a admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:53.552900 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54734f35-1bb7-40ee-849b-6cc7e62a303a admin admin] PATH_INFO: `/v3/projects/db87d8b3358945aa88acbdc4246917ae` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:53.554133 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54734f35-1bb7-40ee-849b-6cc7e62a303a admin admin] RBAC: Authorizing `identity:delete_project(project_id=db87d8b3358945aa88acbdc4246917ae)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:53.556131 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54734f35-1bb7-40ee-849b-6cc7e62a303a admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:53.606136 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-54734f35-1bb7-40ee-849b-6cc7e62a303a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db87d8b3358945aa88acbdc4246917ae', 'request_id': 'req-54734f35-1bb7-40ee-849b-6cc7e62a303a'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:53.612700 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-54734f35-1bb7-40ee-849b-6cc7e62a303a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db87d8b3358945aa88acbdc4246917ae', 'request_id': 'req-54734f35-1bb7-40ee-849b-6cc7e62a303a'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:53.622771 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-54734f35-1bb7-40ee-849b-6cc7e62a303a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db87d8b3358945aa88acbdc4246917ae', 'request_id': 'req-54734f35-1bb7-40ee-849b-6cc7e62a303a'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:53.637806 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-54734f35-1bb7-40ee-849b-6cc7e62a303a admin admin] The token cache is being invalidate because project db87d8b3358945aa88acbdc4246917ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:19:53.638083 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-54734f35-1bb7-40ee-849b-6cc7e62a303a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:53.638834 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-54734f35-1bb7-40ee-849b-6cc7e62a303a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:53.640582 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1381/2769] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:19:53 2022] DELETE /identity/v3/projects/db87d8b3358945aa88acbdc4246917ae => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:19:53.645963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3c92b-9471-4bcf-a67f-cf278e356084 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:53.646270 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3c92b-9471-4bcf-a67f-cf278e356084 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:53.646529 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3c92b-9471-4bcf-a67f-cf278e356084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:53.670547 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-3ec3c92b-9471-4bcf-a67f-cf278e356084 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:19:53.671821 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3ec3c92b-9471-4bcf-a67f-cf278e356084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:19:53.734889 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1389/2770] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:19:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:19:53.743141 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb789f4-f2e8-43b6-bb9c-bff07e345d6e None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:53.743644 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb789f4-f2e8-43b6-bb9c-bff07e345d6e None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:53.743929 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb789f4-f2e8-43b6-bb9c-bff07e345d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:53.772019 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-0cb789f4-f2e8-43b6-bb9c-bff07e345d6e None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:19:53.773303 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0cb789f4-f2e8-43b6-bb9c-bff07e345d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:19:53.782256 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1382/2771] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:19:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:19:53.786859 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3c92b-9471-4bcf-a67f-cf278e356084 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:53.798434 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3c92b-9471-4bcf-a67f-cf278e356084 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:53.803649 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70864981-c64f-4c78-a066-200edd6ff327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:53.804233 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70864981-c64f-4c78-a066-200edd6ff327 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:53.804608 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70864981-c64f-4c78-a066-200edd6ff327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:53.804939 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70864981-c64f-4c78-a066-200edd6ff327 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:53.805782 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70864981-c64f-4c78-a066-200edd6ff327 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:19:53.806327 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70864981-c64f-4c78-a066-200edd6ff327 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:53.808153 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70864981-c64f-4c78-a066-200edd6ff327 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:53.822572 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1390/2772] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:19:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:19:53.827847 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb789f4-f2e8-43b6-bb9c-bff07e345d6e None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:53.845969 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb789f4-f2e8-43b6-bb9c-bff07e345d6e None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:53.854180 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3217dd1-67b5-4449-812a-c2bfe1951551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:53.855223 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3217dd1-67b5-4449-812a-c2bfe1951551 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:53.855727 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3217dd1-67b5-4449-812a-c2bfe1951551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:53.856374 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3217dd1-67b5-4449-812a-c2bfe1951551 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:53.858836 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3217dd1-67b5-4449-812a-c2bfe1951551 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:53.861635 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3217dd1-67b5-4449-812a-c2bfe1951551 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:53.915962 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1383/2773] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:19:53 2022] POST /identity/v3/projects => generated 363 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:19:53.921208 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70864981-c64f-4c78-a066-200edd6ff327 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:53.934631 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70864981-c64f-4c78-a066-200edd6ff327 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:53.940249 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:19:54.009937 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3254a7e-f3bb-4abd-8915-ebc2f23e7b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:54.011144 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3254a7e-f3bb-4abd-8915-ebc2f23e7b8f admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:54.011744 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3254a7e-f3bb-4abd-8915-ebc2f23e7b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:54.012237 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3254a7e-f3bb-4abd-8915-ebc2f23e7b8f admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:54.014118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3254a7e-f3bb-4abd-8915-ebc2f23e7b8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:54.017604 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3254a7e-f3bb-4abd-8915-ebc2f23e7b8f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:54.047604 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1391/2774] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:19:53 2022] POST /identity/v3/users => generated 282 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:19:54.052093 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3217dd1-67b5-4449-812a-c2bfe1951551 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:54.068489 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3217dd1-67b5-4449-812a-c2bfe1951551 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:54.074174 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3dac21-391e-410f-9ec6-ff1acfca4352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:54.074915 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3dac21-391e-410f-9ec6-ff1acfca4352 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:54.075141 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3dac21-391e-410f-9ec6-ff1acfca4352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:54.075353 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3dac21-391e-410f-9ec6-ff1acfca4352 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:54.076111 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3dac21-391e-410f-9ec6-ff1acfca4352 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:54.078934 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3dac21-391e-410f-9ec6-ff1acfca4352 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:54.089016 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1384/2775] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:19:54.093365 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3254a7e-f3bb-4abd-8915-ebc2f23e7b8f admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:54.104938 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3254a7e-f3bb-4abd-8915-ebc2f23e7b8f admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:54.111490 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b77c706-497b-4261-a4f6-87219aa01cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:54.112416 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77c706-497b-4261-a4f6-87219aa01cd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:54.112710 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77c706-497b-4261-a4f6-87219aa01cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:54.112983 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77c706-497b-4261-a4f6-87219aa01cd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:54.114605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b77c706-497b-4261-a4f6-87219aa01cd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:54.115422 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b77c706-497b-4261-a4f6-87219aa01cd1 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:54.130091 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1392/2776] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:19:54.135999 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3dac21-391e-410f-9ec6-ff1acfca4352 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:54.149859 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3dac21-391e-410f-9ec6-ff1acfca4352 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:54.156447 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b52166-7981-4579-9cb9-c535a98d9a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:54.157657 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b52166-7981-4579-9cb9-c535a98d9a68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:54.157962 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b52166-7981-4579-9cb9-c535a98d9a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:54.158292 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b52166-7981-4579-9cb9-c535a98d9a68 admin admin] PATH_INFO: `/v3/projects/57977983716d49b4b316ed9e848ba72c/users/3bdcefb560444d8e9fda9e7c4a5ad130/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:54.177274 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b52166-7981-4579-9cb9-c535a98d9a68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57977983716d49b4b316ed9e848ba72c, user_id=3bdcefb560444d8e9fda9e7c4a5ad130, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:54.180097 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b52166-7981-4579-9cb9-c535a98d9a68 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:54.195475 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1385/2777] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:19:54 2022] PUT /identity/v3/projects/57977983716d49b4b316ed9e848ba72c/users/3bdcefb560444d8e9fda9e7c4a5ad130/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:19:54.207256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37bf1b7-004d-432a-9c2b-196145cf274e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:54.207534 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37bf1b7-004d-432a-9c2b-196145cf274e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:54.207838 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37bf1b7-004d-432a-9c2b-196145cf274e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:54.236898 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-e37bf1b7-004d-432a-9c2b-196145cf274e None None] MFA Rules not processed for user `3bdcefb560444d8e9fda9e7c4a5ad130`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:19:54.238143 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e37bf1b7-004d-432a-9c2b-196145cf274e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:19:54.308901 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1393/2778] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:19:54 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:19:54.314280 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b52166-7981-4579-9cb9-c535a98d9a68 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:54.336184 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:19:54.393983 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b52166-7981-4579-9cb9-c535a98d9a68 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:54.399945 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d6c520-c302-423b-be69-d892ec131b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:54.400658 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6c520-c302-423b-be69-d892ec131b50 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:54.400920 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6c520-c302-423b-be69-d892ec131b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:54.401196 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6c520-c302-423b-be69-d892ec131b50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:54.401877 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d6c520-c302-423b-be69-d892ec131b50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:54.403868 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d6c520-c302-423b-be69-d892ec131b50 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:54.433387 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1386/2779] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:19:54 2022] POST /identity/v3/projects => generated 363 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:19:54.438059 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37bf1b7-004d-432a-9c2b-196145cf274e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:54.450688 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37bf1b7-004d-432a-9c2b-196145cf274e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:54.515145 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81262ad-2599-4633-b299-b4901c381251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:54.515787 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81262ad-2599-4633-b299-b4901c381251 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:54.516070 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81262ad-2599-4633-b299-b4901c381251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:54.516394 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81262ad-2599-4633-b299-b4901c381251 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:54.517066 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81262ad-2599-4633-b299-b4901c381251 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:54.519078 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81262ad-2599-4633-b299-b4901c381251 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:54.547671 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1394/2780] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:19:54 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:19:54.552222 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d6c520-c302-423b-be69-d892ec131b50 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:54.564978 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d6c520-c302-423b-be69-d892ec131b50 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:54.573103 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9bb58-4079-45c7-92ca-cda34b563766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:54.573751 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9bb58-4079-45c7-92ca-cda34b563766 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:54.574085 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9bb58-4079-45c7-92ca-cda34b563766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:54.574361 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9bb58-4079-45c7-92ca-cda34b563766 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:54.575090 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd9bb58-4079-45c7-92ca-cda34b563766 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:54.576752 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd9bb58-4079-45c7-92ca-cda34b563766 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:54.588301 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1387/2781] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:19:54.592842 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81262ad-2599-4633-b299-b4901c381251 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:54.599902 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:19:54.604712 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81262ad-2599-4633-b299-b4901c381251 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:54.610847 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ddd56-b893-4e1f-afa7-27b20c6ec0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:54.611542 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ddd56-b893-4e1f-afa7-27b20c6ec0c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:54.611849 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ddd56-b893-4e1f-afa7-27b20c6ec0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:54.612113 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ddd56-b893-4e1f-afa7-27b20c6ec0c2 admin admin] PATH_INFO: `/v3/projects/ce1c49c382e242159f374482062353e4/users/887b072b3afe48d189a56398a4ad3e4d/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:54.637124 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ddd56-b893-4e1f-afa7-27b20c6ec0c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce1c49c382e242159f374482062353e4, user_id=887b072b3afe48d189a56398a4ad3e4d, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:54.639838 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ddd56-b893-4e1f-afa7-27b20c6ec0c2 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:54.655143 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1395/2782] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:19:54 2022] PUT /identity/v3/projects/ce1c49c382e242159f374482062353e4/users/887b072b3afe48d189a56398a4ad3e4d/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:19:54.661150 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9bb58-4079-45c7-92ca-cda34b563766 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:54.676947 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9bb58-4079-45c7-92ca-cda34b563766 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:54.745209 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1438f00c-63a0-4cdb-85c0-f73ab29c953d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:54.746347 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438f00c-63a0-4cdb-85c0-f73ab29c953d admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:54.746888 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438f00c-63a0-4cdb-85c0-f73ab29c953d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:54.747350 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438f00c-63a0-4cdb-85c0-f73ab29c953d admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:54.748749 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1438f00c-63a0-4cdb-85c0-f73ab29c953d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:54.751893 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1438f00c-63a0-4cdb-85c0-f73ab29c953d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:54.769961 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1388/2783] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:19:54.774280 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ddd56-b893-4e1f-afa7-27b20c6ec0c2 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:54.786181 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ddd56-b893-4e1f-afa7-27b20c6ec0c2 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:54.791289 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57739af-1433-4b68-bb71-0431d04a7533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:54.793825 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57739af-1433-4b68-bb71-0431d04a7533 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:54.794120 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57739af-1433-4b68-bb71-0431d04a7533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:54.794403 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57739af-1433-4b68-bb71-0431d04a7533 admin admin] PATH_INFO: `/v3/projects/ce1c49c382e242159f374482062353e4/users/887b072b3afe48d189a56398a4ad3e4d/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:54.796644 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57739af-1433-4b68-bb71-0431d04a7533 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce1c49c382e242159f374482062353e4, user_id=887b072b3afe48d189a56398a4ad3e4d, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:54.799334 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57739af-1433-4b68-bb71-0431d04a7533 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:54.811498 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1396/2784] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:19:54 2022] PUT /identity/v3/projects/ce1c49c382e242159f374482062353e4/users/887b072b3afe48d189a56398a4ad3e4d/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:19:54.823049 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20589a-4b2e-45e5-a770-c35878ee45a3 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:54.823481 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20589a-4b2e-45e5-a770-c35878ee45a3 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:54.823938 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20589a-4b2e-45e5-a770-c35878ee45a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:54.858665 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-2a20589a-4b2e-45e5-a770-c35878ee45a3 None None] MFA Rules not processed for user `887b072b3afe48d189a56398a4ad3e4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:19:54.859899 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-2a20589a-4b2e-45e5-a770-c35878ee45a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:19:54.935803 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1389/2785] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:19:54 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:19:54.948087 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57739af-1433-4b68-bb71-0431d04a7533 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:54.949873 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c57739af-1433-4b68-bb71-0431d04a7533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:19:55.013681 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57739af-1433-4b68-bb71-0431d04a7533 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:55.017792 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91096874-b656-4501-a99d-3a34e0411c0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:55.018558 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91096874-b656-4501-a99d-3a34e0411c0a service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:55.018826 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91096874-b656-4501-a99d-3a34e0411c0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:55.019094 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91096874-b656-4501-a99d-3a34e0411c0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:55.029846 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91096874-b656-4501-a99d-3a34e0411c0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:55.031298 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91096874-b656-4501-a99d-3a34e0411c0a service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:55.041766 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1397/2786] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:19:54 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:19:55.131462 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20589a-4b2e-45e5-a770-c35878ee45a3 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:55.142373 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20589a-4b2e-45e5-a770-c35878ee45a3 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:55.147017 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea175d15-9e34-49f2-bf18-e965ed2f2ad9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:55.147639 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea175d15-9e34-49f2-bf18-e965ed2f2ad9 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:55.147917 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea175d15-9e34-49f2-bf18-e965ed2f2ad9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:55.148185 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea175d15-9e34-49f2-bf18-e965ed2f2ad9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:55.158055 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:19:55.209393 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea175d15-9e34-49f2-bf18-e965ed2f2ad9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:55.210832 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea175d15-9e34-49f2-bf18-e965ed2f2ad9 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:55.220037 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1390/2787] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:19:55 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:19:58.039528 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91096874-b656-4501-a99d-3a34e0411c0a service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:58.109662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91096874-b656-4501-a99d-3a34e0411c0a service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:58.114374 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc1778-bb33-4a2d-9b38-3bb68f33a628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:58.115035 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc1778-bb33-4a2d-9b38-3bb68f33a628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:58.115314 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc1778-bb33-4a2d-9b38-3bb68f33a628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:58.115585 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc1778-bb33-4a2d-9b38-3bb68f33a628 admin admin] PATH_INFO: `/v3/users/3bdcefb560444d8e9fda9e7c4a5ad130` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:58.116778 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc1778-bb33-4a2d-9b38-3bb68f33a628 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bdcefb560444d8e9fda9e7c4a5ad130)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:58.118794 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc1778-bb33-4a2d-9b38-3bb68f33a628 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:58.179581 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-68bc1778-bb33-4a2d-9b38-3bb68f33a628 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bdcefb560444d8e9fda9e7c4a5ad130'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:58.180458 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-68bc1778-bb33-4a2d-9b38-3bb68f33a628 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bdcefb560444d8e9fda9e7c4a5ad130'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:58.196446 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-68bc1778-bb33-4a2d-9b38-3bb68f33a628 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bdcefb560444d8e9fda9e7c4a5ad130'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:58.202114 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:19:58.204499 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-68bc1778-bb33-4a2d-9b38-3bb68f33a628 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bdcefb560444d8e9fda9e7c4a5ad130'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:58.205736 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-68bc1778-bb33-4a2d-9b38-3bb68f33a628 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bdcefb560444d8e9fda9e7c4a5ad130'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:58.221189 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1398/2788] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:19:58 2022] DELETE /identity/v3/users/3bdcefb560444d8e9fda9e7c4a5ad130 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:19:58.225932 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d6c8d-2787-423f-821c-53e4d1c653af None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:58.226769 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d6c8d-2787-423f-821c-53e4d1c653af None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:58.226769 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d6c8d-2787-423f-821c-53e4d1c653af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:58.246291 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-704d6c8d-2787-423f-821c-53e4d1c653af None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:19:58.247491 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-704d6c8d-2787-423f-821c-53e4d1c653af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:19:58.326241 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1391/2789] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:19:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:19:58.336963 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc1778-bb33-4a2d-9b38-3bb68f33a628 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:58.338691 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-68bc1778-bb33-4a2d-9b38-3bb68f33a628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:19:58.404170 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc1778-bb33-4a2d-9b38-3bb68f33a628 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:58.409229 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa9867-6567-4e57-a19c-33016a4fe7ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:58.409864 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa9867-6567-4e57-a19c-33016a4fe7ca service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:58.410144 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa9867-6567-4e57-a19c-33016a4fe7ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:58.410413 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa9867-6567-4e57-a19c-33016a4fe7ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:58.421337 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfa9867-6567-4e57-a19c-33016a4fe7ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:58.422893 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfa9867-6567-4e57-a19c-33016a4fe7ca service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:58.436256 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1399/2790] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:19:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:19:58.764130 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d6c8d-2787-423f-821c-53e4d1c653af None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:58.765890 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-704d6c8d-2787-423f-821c-53e4d1c653af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:19:58.783486 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d6c8d-2787-423f-821c-53e4d1c653af None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:58.788786 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cfeb1e-fe55-46d7-9f37-2dbb23d84dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:58.789453 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfeb1e-fe55-46d7-9f37-2dbb23d84dda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:58.789734 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfeb1e-fe55-46d7-9f37-2dbb23d84dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:58.790001 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfeb1e-fe55-46d7-9f37-2dbb23d84dda admin admin] PATH_INFO: `/v3/projects/57977983716d49b4b316ed9e848ba72c` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:58.791405 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cfeb1e-fe55-46d7-9f37-2dbb23d84dda admin admin] RBAC: Authorizing `identity:delete_project(project_id=57977983716d49b4b316ed9e848ba72c)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:58.793388 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cfeb1e-fe55-46d7-9f37-2dbb23d84dda admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:58.845278 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-31cfeb1e-fe55-46d7-9f37-2dbb23d84dda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57977983716d49b4b316ed9e848ba72c', 'request_id': 'req-31cfeb1e-fe55-46d7-9f37-2dbb23d84dda'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:58.852801 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-31cfeb1e-fe55-46d7-9f37-2dbb23d84dda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57977983716d49b4b316ed9e848ba72c', 'request_id': 'req-31cfeb1e-fe55-46d7-9f37-2dbb23d84dda'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:58.862221 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-31cfeb1e-fe55-46d7-9f37-2dbb23d84dda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57977983716d49b4b316ed9e848ba72c', 'request_id': 'req-31cfeb1e-fe55-46d7-9f37-2dbb23d84dda'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:58.873541 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-31cfeb1e-fe55-46d7-9f37-2dbb23d84dda admin admin] The token cache is being invalidate because project 57977983716d49b4b316ed9e848ba72c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:19:58.873829 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-31cfeb1e-fe55-46d7-9f37-2dbb23d84dda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:58.874480 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-31cfeb1e-fe55-46d7-9f37-2dbb23d84dda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:58.876011 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1392/2791] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:19:58 2022] DELETE /identity/v3/projects/57977983716d49b4b316ed9e848ba72c => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:19:58.880901 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa9867-6567-4e57-a19c-33016a4fe7ca service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:58.891305 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa9867-6567-4e57-a19c-33016a4fe7ca service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:58.951034 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4cf14-792b-4b77-8da3-c40f65bae88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:58.951591 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4cf14-792b-4b77-8da3-c40f65bae88f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:58.951866 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4cf14-792b-4b77-8da3-c40f65bae88f admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:58.952141 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4cf14-792b-4b77-8da3-c40f65bae88f admin admin] PATH_INFO: `/v3/users/887b072b3afe48d189a56398a4ad3e4d` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:58.953335 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4cf14-792b-4b77-8da3-c40f65bae88f admin admin] RBAC: Authorizing `identity:delete_user(user_id=887b072b3afe48d189a56398a4ad3e4d)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:58.955298 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4cf14-792b-4b77-8da3-c40f65bae88f admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:59.010599 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a1b4cf14-792b-4b77-8da3-c40f65bae88f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '887b072b3afe48d189a56398a4ad3e4d'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:59.011351 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a1b4cf14-792b-4b77-8da3-c40f65bae88f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '887b072b3afe48d189a56398a4ad3e4d'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:59.026916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a1b4cf14-792b-4b77-8da3-c40f65bae88f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '887b072b3afe48d189a56398a4ad3e4d'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:59.035471 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a1b4cf14-792b-4b77-8da3-c40f65bae88f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '887b072b3afe48d189a56398a4ad3e4d'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:59.036211 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a1b4cf14-792b-4b77-8da3-c40f65bae88f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '887b072b3afe48d189a56398a4ad3e4d'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:59.052662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1400/2792] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:19:58 2022] DELETE /identity/v3/users/887b072b3afe48d189a56398a4ad3e4d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:19:59.358637 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cfeb1e-fe55-46d7-9f37-2dbb23d84dda admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:59.360197 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-31cfeb1e-fe55-46d7-9f37-2dbb23d84dda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:19:59.373931 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cfeb1e-fe55-46d7-9f37-2dbb23d84dda admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:59.411312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:19:59.444687 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc06382-9b80-403d-b98d-5fba3ab8f4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:59.445354 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc06382-9b80-403d-b98d-5fba3ab8f4cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:59.445641 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc06382-9b80-403d-b98d-5fba3ab8f4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:59.445929 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc06382-9b80-403d-b98d-5fba3ab8f4cc admin admin] PATH_INFO: `/v3/projects/ce1c49c382e242159f374482062353e4` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:59.447144 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc06382-9b80-403d-b98d-5fba3ab8f4cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce1c49c382e242159f374482062353e4)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:59.449234 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc06382-9b80-403d-b98d-5fba3ab8f4cc admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:59.515004 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-7bc06382-9b80-403d-b98d-5fba3ab8f4cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce1c49c382e242159f374482062353e4', 'request_id': 'req-7bc06382-9b80-403d-b98d-5fba3ab8f4cc'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:59.521690 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-7bc06382-9b80-403d-b98d-5fba3ab8f4cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce1c49c382e242159f374482062353e4', 'request_id': 'req-7bc06382-9b80-403d-b98d-5fba3ab8f4cc'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:59.532034 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-7bc06382-9b80-403d-b98d-5fba3ab8f4cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce1c49c382e242159f374482062353e4', 'request_id': 'req-7bc06382-9b80-403d-b98d-5fba3ab8f4cc'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:59.545991 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-7bc06382-9b80-403d-b98d-5fba3ab8f4cc admin admin] The token cache is being invalidate because project ce1c49c382e242159f374482062353e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:19:59.546395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-7bc06382-9b80-403d-b98d-5fba3ab8f4cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:59.547032 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-7bc06382-9b80-403d-b98d-5fba3ab8f4cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:19:59.548710 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1393/2793] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:19:59 2022] DELETE /identity/v3/projects/ce1c49c382e242159f374482062353e4 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:19:59.553888 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2320a-c368-483a-b209-c44a5dd1cb06 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:59.554177 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2320a-c368-483a-b209-c44a5dd1cb06 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:59.554447 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2320a-c368-483a-b209-c44a5dd1cb06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:59.575746 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-8ee2320a-c368-483a-b209-c44a5dd1cb06 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:19:59.577021 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-8ee2320a-c368-483a-b209-c44a5dd1cb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:19:59.582880 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:19:59.646861 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1401/2794] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:19:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:19:59.654904 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2f7e2-2402-435d-ad4e-25d02a37fb72 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:59.655204 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2f7e2-2402-435d-ad4e-25d02a37fb72 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:59.655475 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2f7e2-2402-435d-ad4e-25d02a37fb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:59.677623 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-28a2f7e2-2402-435d-ad4e-25d02a37fb72 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:19:59.678868 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-28a2f7e2-2402-435d-ad4e-25d02a37fb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:19:59.688346 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1394/2795] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:19:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:19:59.692921 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2320a-c368-483a-b209-c44a5dd1cb06 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:59.703269 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2320a-c368-483a-b209-c44a5dd1cb06 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:59.713353 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9d316-becc-46b8-b31c-a2c9118357d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:59.714039 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9d316-becc-46b8-b31c-a2c9118357d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:59.714318 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9d316-becc-46b8-b31c-a2c9118357d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:59.714598 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9d316-becc-46b8-b31c-a2c9118357d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:59.715313 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b9d316-becc-46b8-b31c-a2c9118357d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:19:59.715617 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b9d316-becc-46b8-b31c-a2c9118357d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:59.717279 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b9d316-becc-46b8-b31c-a2c9118357d4 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:59.730386 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1402/2796] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:19:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:19:59.744154 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2f7e2-2402-435d-ad4e-25d02a37fb72 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:59.753904 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2f7e2-2402-435d-ad4e-25d02a37fb72 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:59.759472 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94b860-c277-4937-94b2-42557016f6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:59.760110 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94b860-c277-4937-94b2-42557016f6d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:59.760446 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94b860-c277-4937-94b2-42557016f6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:59.760717 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94b860-c277-4937-94b2-42557016f6d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:59.761375 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f94b860-c277-4937-94b2-42557016f6d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:59.763311 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f94b860-c277-4937-94b2-42557016f6d8 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:59.805025 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1395/2797] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:19:59 2022] POST /identity/v3/projects => generated 381 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:19:59.809580 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9d316-becc-46b8-b31c-a2c9118357d4 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:59.822888 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9d316-becc-46b8-b31c-a2c9118357d4 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:59.892038 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad25a0a9-90e4-42be-a97e-ae512d67b6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:59.892703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25a0a9-90e4-42be-a97e-ae512d67b6c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:59.892988 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25a0a9-90e4-42be-a97e-ae512d67b6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:59.893264 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25a0a9-90e4-42be-a97e-ae512d67b6c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:59.893986 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad25a0a9-90e4-42be-a97e-ae512d67b6c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:59.895933 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad25a0a9-90e4-42be-a97e-ae512d67b6c9 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:59.927710 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1403/2798] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:19:59 2022] POST /identity/v3/users => generated 291 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:19:59.932766 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94b860-c277-4937-94b2-42557016f6d8 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:59.947793 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94b860-c277-4937-94b2-42557016f6d8 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:59.954644 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600a19a-353a-438d-96b4-72f60cb42e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:59.959801 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600a19a-353a-438d-96b4-72f60cb42e9f admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:59.960160 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600a19a-353a-438d-96b4-72f60cb42e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:59.960455 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600a19a-353a-438d-96b4-72f60cb42e9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:59.961385 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600a19a-353a-438d-96b4-72f60cb42e9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:59.963228 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600a19a-353a-438d-96b4-72f60cb42e9f admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:59.974692 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1396/2799] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:19:59.979778 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad25a0a9-90e4-42be-a97e-ae512d67b6c9 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:59.992037 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad25a0a9-90e4-42be-a97e-ae512d67b6c9 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:59.998189 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb912e4-2e3f-41fc-8821-e6ddd7cd6bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:59.998840 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb912e4-2e3f-41fc-8821-e6ddd7cd6bcf admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:59.999286 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb912e4-2e3f-41fc-8821-e6ddd7cd6bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:59.999741 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb912e4-2e3f-41fc-8821-e6ddd7cd6bcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:00.000856 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb912e4-2e3f-41fc-8821-e6ddd7cd6bcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:00.003355 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb912e4-2e3f-41fc-8821-e6ddd7cd6bcf admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:00.018109 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1404/2800] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:00.023397 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600a19a-353a-438d-96b4-72f60cb42e9f admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:00.035657 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600a19a-353a-438d-96b4-72f60cb42e9f admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:00.041054 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22a027-870d-48de-a9be-298d3509c0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:00.041744 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22a027-870d-48de-a9be-298d3509c0f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:00.042047 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22a027-870d-48de-a9be-298d3509c0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:00.042303 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22a027-870d-48de-a9be-298d3509c0f6 admin admin] PATH_INFO: `/v3/projects/1188b1283a7e4210885a9eb3b767b4d8/users/576c867124e84cb18193914abc62a789/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:00.059738 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22a027-870d-48de-a9be-298d3509c0f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1188b1283a7e4210885a9eb3b767b4d8, user_id=576c867124e84cb18193914abc62a789, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:00.062695 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22a027-870d-48de-a9be-298d3509c0f6 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:00.077634 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1397/2801] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:00 2022] PUT /identity/v3/projects/1188b1283a7e4210885a9eb3b767b4d8/users/576c867124e84cb18193914abc62a789/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:00.150529 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35c825-2694-4f9b-bbb2-1206e4f6e5a9 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:00.151276 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35c825-2694-4f9b-bbb2-1206e4f6e5a9 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:00.151514 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35c825-2694-4f9b-bbb2-1206e4f6e5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:00.193225 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-2c35c825-2694-4f9b-bbb2-1206e4f6e5a9 None None] MFA Rules not processed for user `576c867124e84cb18193914abc62a789`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:00.194487 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-2c35c825-2694-4f9b-bbb2-1206e4f6e5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:00.267978 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1405/2802] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:20:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:00.273297 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22a027-870d-48de-a9be-298d3509c0f6 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:00.344870 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22a027-870d-48de-a9be-298d3509c0f6 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:00.351126 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dbe095-c18f-4242-b545-618cdf9e6cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:00.351794 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dbe095-c18f-4242-b545-618cdf9e6cec admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:00.352074 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dbe095-c18f-4242-b545-618cdf9e6cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:00.352455 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dbe095-c18f-4242-b545-618cdf9e6cec admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:00.353203 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dbe095-c18f-4242-b545-618cdf9e6cec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:00.355121 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dbe095-c18f-4242-b545-618cdf9e6cec admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:00.361045 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:00.386744 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1398/2803] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:00 2022] POST /identity/v3/projects => generated 381 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:00.393329 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35c825-2694-4f9b-bbb2-1206e4f6e5a9 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:00.408198 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35c825-2694-4f9b-bbb2-1206e4f6e5a9 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:00.413623 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:00.479272 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ef565-71bc-4423-a791-19d63e5a1106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:00.479914 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ef565-71bc-4423-a791-19d63e5a1106 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:00.480194 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ef565-71bc-4423-a791-19d63e5a1106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:00.480557 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ef565-71bc-4423-a791-19d63e5a1106 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:00.481231 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ef565-71bc-4423-a791-19d63e5a1106 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:00.483210 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ef565-71bc-4423-a791-19d63e5a1106 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:00.509760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1406/2804] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:00 2022] POST /identity/v3/users => generated 297 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:00.513585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dbe095-c18f-4242-b545-618cdf9e6cec admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:00.528427 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dbe095-c18f-4242-b545-618cdf9e6cec admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:00.533726 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74241f7-a63d-4e37-b0d2-de20068b3c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:00.534351 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74241f7-a63d-4e37-b0d2-de20068b3c2a admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:00.534645 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74241f7-a63d-4e37-b0d2-de20068b3c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:00.534913 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74241f7-a63d-4e37-b0d2-de20068b3c2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:00.535633 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74241f7-a63d-4e37-b0d2-de20068b3c2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:00.537289 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74241f7-a63d-4e37-b0d2-de20068b3c2a admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:00.548721 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1399/2805] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:00.553055 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ef565-71bc-4423-a791-19d63e5a1106 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:00.566802 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ef565-71bc-4423-a791-19d63e5a1106 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:00.571662 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d2700-30fd-44b9-9817-7ba3fc7c37e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:00.572397 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d2700-30fd-44b9-9817-7ba3fc7c37e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:00.572676 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d2700-30fd-44b9-9817-7ba3fc7c37e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:00.572949 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d2700-30fd-44b9-9817-7ba3fc7c37e8 admin admin] PATH_INFO: `/v3/projects/fef97eb490104876b65d08293255ddf5/users/221799880b9f472a879c371653f36af6/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:00.589200 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6d2700-30fd-44b9-9817-7ba3fc7c37e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fef97eb490104876b65d08293255ddf5, user_id=221799880b9f472a879c371653f36af6, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:00.591868 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6d2700-30fd-44b9-9817-7ba3fc7c37e8 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:00.609697 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1407/2806] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:00 2022] PUT /identity/v3/projects/fef97eb490104876b65d08293255ddf5/users/221799880b9f472a879c371653f36af6/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:00.615978 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74241f7-a63d-4e37-b0d2-de20068b3c2a admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:00.627790 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74241f7-a63d-4e37-b0d2-de20068b3c2a admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:00.694756 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dbd46f-19fd-45d8-8bcf-37db1156e89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:00.695399 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dbd46f-19fd-45d8-8bcf-37db1156e89b admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:00.695660 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dbd46f-19fd-45d8-8bcf-37db1156e89b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:00.695996 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dbd46f-19fd-45d8-8bcf-37db1156e89b admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:00.696893 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dbd46f-19fd-45d8-8bcf-37db1156e89b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:00.698736 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dbd46f-19fd-45d8-8bcf-37db1156e89b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:00.710721 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1400/2807] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:00 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:00.715350 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d2700-30fd-44b9-9817-7ba3fc7c37e8 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:00.728407 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d2700-30fd-44b9-9817-7ba3fc7c37e8 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:00.734117 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1916154-8529-46de-ab69-0af4fd31f054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:00.734800 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1916154-8529-46de-ab69-0af4fd31f054 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:00.735079 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1916154-8529-46de-ab69-0af4fd31f054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:00.735352 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1916154-8529-46de-ab69-0af4fd31f054 admin admin] PATH_INFO: `/v3/projects/fef97eb490104876b65d08293255ddf5/users/221799880b9f472a879c371653f36af6/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:00.737669 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1916154-8529-46de-ab69-0af4fd31f054 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fef97eb490104876b65d08293255ddf5, user_id=221799880b9f472a879c371653f36af6, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:00.740399 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1916154-8529-46de-ab69-0af4fd31f054 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:00.752342 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1408/2808] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:00 2022] PUT /identity/v3/projects/fef97eb490104876b65d08293255ddf5/users/221799880b9f472a879c371653f36af6/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:20:00.762959 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c4cfe-816b-49cb-bad7-3735d96cf699 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:00.763321 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c4cfe-816b-49cb-bad7-3735d96cf699 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:00.763675 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c4cfe-816b-49cb-bad7-3735d96cf699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:00.813779 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-e80c4cfe-816b-49cb-bad7-3735d96cf699 None None] MFA Rules not processed for user `221799880b9f472a879c371653f36af6`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:00.815797 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-e80c4cfe-816b-49cb-bad7-3735d96cf699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:00.905350 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1401/2809] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:20:00 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:00.920251 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1916154-8529-46de-ab69-0af4fd31f054 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:00.922058 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a1916154-8529-46de-ab69-0af4fd31f054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:00.986230 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1916154-8529-46de-ab69-0af4fd31f054 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:00.990188 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2146e-f403-4917-9017-bee5fe96cccb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:00.990843 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2146e-f403-4917-9017-bee5fe96cccb service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:00.991118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2146e-f403-4917-9017-bee5fe96cccb service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:00.991391 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2146e-f403-4917-9017-bee5fe96cccb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:01.001742 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca2146e-f403-4917-9017-bee5fe96cccb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:01.003298 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca2146e-f403-4917-9017-bee5fe96cccb service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:01.014424 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1409/2810] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:00 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:01.389843 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80c4cfe-816b-49cb-bad7-3735d96cf699 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:01.403059 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80c4cfe-816b-49cb-bad7-3735d96cf699 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:01.408990 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194ef81-b6a2-4b41-bf59-50a50ee3fd17 tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '221799880b9f472a879c371653f36af6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fef97eb490104876b65d08293255ddf5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:01.409895 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194ef81-b6a2-4b41-bf59-50a50ee3fd17 tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:01.410335 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194ef81-b6a2-4b41-bf59-50a50ee3fd17 tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:01.410787 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194ef81-b6a2-4b41-bf59-50a50ee3fd17 tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:01.411674 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194ef81-b6a2-4b41-bf59-50a50ee3fd17 tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:20:01.411971 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194ef81-b6a2-4b41-bf59-50a50ee3fd17 tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:01.414000 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194ef81-b6a2-4b41-bf59-50a50ee3fd17 tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:01.429385 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1402/2811] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:20:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:20:01.434010 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2146e-f403-4917-9017-bee5fe96cccb service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:01.443987 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2146e-f403-4917-9017-bee5fe96cccb service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:01.449798 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bfa7c5-2984-40f9-b42e-e0bb5124ccfb tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '221799880b9f472a879c371653f36af6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fef97eb490104876b65d08293255ddf5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:01.450909 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfa7c5-2984-40f9-b42e-e0bb5124ccfb tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:01.451376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfa7c5-2984-40f9-b42e-e0bb5124ccfb tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:01.451674 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfa7c5-2984-40f9-b42e-e0bb5124ccfb tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:01.452394 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bfa7c5-2984-40f9-b42e-e0bb5124ccfb tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:01.454507 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bfa7c5-2984-40f9-b42e-e0bb5124ccfb tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:01.492196 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1410/2812] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:01 2022] POST /identity/v3/projects => generated 353 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:01.497271 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194ef81-b6a2-4b41-bf59-50a50ee3fd17 tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:01.513755 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194ef81-b6a2-4b41-bf59-50a50ee3fd17 tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:01.571444 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:01.590827 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7fa071-0feb-4476-b974-86339ccd75dc tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '221799880b9f472a879c371653f36af6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fef97eb490104876b65d08293255ddf5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:01.591451 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fa071-0feb-4476-b974-86339ccd75dc tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:01.591730 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fa071-0feb-4476-b974-86339ccd75dc tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:01.591998 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fa071-0feb-4476-b974-86339ccd75dc tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:01.592740 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7fa071-0feb-4476-b974-86339ccd75dc tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:20:01.593044 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7fa071-0feb-4476-b974-86339ccd75dc tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:01.594638 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7fa071-0feb-4476-b974-86339ccd75dc tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:01.607262 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1403/2813] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:20:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:20:01.645664 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bfa7c5-2984-40f9-b42e-e0bb5124ccfb tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:01.657004 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bfa7c5-2984-40f9-b42e-e0bb5124ccfb tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:01.662538 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cef430-de20-440c-b93d-343af8a86edc tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '221799880b9f472a879c371653f36af6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fef97eb490104876b65d08293255ddf5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:01.663200 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cef430-de20-440c-b93d-343af8a86edc tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:01.663485 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cef430-de20-440c-b93d-343af8a86edc tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:01.663760 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cef430-de20-440c-b93d-343af8a86edc tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] PATH_INFO: `/v3/projects/9ead4787623f44c6907bfc84730e8362` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:01.664977 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cef430-de20-440c-b93d-343af8a86edc tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] RBAC: Authorizing `identity:delete_project(project_id=9ead4787623f44c6907bfc84730e8362)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:01.667017 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cef430-de20-440c-b93d-343af8a86edc tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:01.714674 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=92,keystone:INSERT=6,keystone:DELETE=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:01.727229 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-29cef430-de20-440c-b93d-343af8a86edc tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ead4787623f44c6907bfc84730e8362', 'request_id': 'req-29cef430-de20-440c-b93d-343af8a86edc'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:01.732892 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-29cef430-de20-440c-b93d-343af8a86edc tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ead4787623f44c6907bfc84730e8362', 'request_id': 'req-29cef430-de20-440c-b93d-343af8a86edc'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:01.742307 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-29cef430-de20-440c-b93d-343af8a86edc tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ead4787623f44c6907bfc84730e8362', 'request_id': 'req-29cef430-de20-440c-b93d-343af8a86edc'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:01.754904 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-29cef430-de20-440c-b93d-343af8a86edc tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] The token cache is being invalidate because project 9ead4787623f44c6907bfc84730e8362 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:20:01.755192 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-29cef430-de20-440c-b93d-343af8a86edc tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:01.755849 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-29cef430-de20-440c-b93d-343af8a86edc tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:01.757490 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1411/2814] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:20:01 2022] DELETE /identity/v3/projects/9ead4787623f44c6907bfc84730e8362 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:20:01.982963 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7fa071-0feb-4476-b974-86339ccd75dc tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:01.985001 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-db7fa071-0feb-4476-b974-86339ccd75dc tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:02.066962 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7fa071-0feb-4476-b974-86339ccd75dc tempest-VolumeQuotaClassesTest-1216557967 tempest-VolumeQuotaClassesTest-1216557967-project-admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:02.125919 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe5708-a778-400a-8283-caf6840f1b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:02.126569 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe5708-a778-400a-8283-caf6840f1b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:02.126844 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe5708-a778-400a-8283-caf6840f1b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:02.127108 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe5708-a778-400a-8283-caf6840f1b73 admin admin] PATH_INFO: `/v3/users/576c867124e84cb18193914abc62a789` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:02.128467 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe5708-a778-400a-8283-caf6840f1b73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=576c867124e84cb18193914abc62a789)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:02.130463 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe5708-a778-400a-8283-caf6840f1b73 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:02.189126 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-96fe5708-a778-400a-8283-caf6840f1b73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '576c867124e84cb18193914abc62a789'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:02.190208 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-96fe5708-a778-400a-8283-caf6840f1b73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '576c867124e84cb18193914abc62a789'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:02.203142 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-96fe5708-a778-400a-8283-caf6840f1b73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '576c867124e84cb18193914abc62a789'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:02.213108 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-96fe5708-a778-400a-8283-caf6840f1b73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '576c867124e84cb18193914abc62a789'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:02.226470 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-96fe5708-a778-400a-8283-caf6840f1b73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '576c867124e84cb18193914abc62a789'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:02.232915 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1404/2815] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:20:01 2022] DELETE /identity/v3/users/576c867124e84cb18193914abc62a789 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:02.237804 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ab88a-f056-41e0-b7d6-26b8ed5e029e None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:02.238246 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ab88a-f056-41e0-b7d6-26b8ed5e029e None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:02.238544 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ab88a-f056-41e0-b7d6-26b8ed5e029e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:02.259188 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-3e7ab88a-f056-41e0-b7d6-26b8ed5e029e None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:02.260571 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3e7ab88a-f056-41e0-b7d6-26b8ed5e029e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:02.324490 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1412/2816] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:20:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:02.334912 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe5708-a778-400a-8283-caf6840f1b73 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:02.336762 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-96fe5708-a778-400a-8283-caf6840f1b73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:02.344264 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:02.407591 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe5708-a778-400a-8283-caf6840f1b73 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:02.413507 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab502d5-5c87-4417-b8ee-289e685be324 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:02.414360 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab502d5-5c87-4417-b8ee-289e685be324 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:02.414656 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab502d5-5c87-4417-b8ee-289e685be324 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:02.414959 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab502d5-5c87-4417-b8ee-289e685be324 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:02.425633 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab502d5-5c87-4417-b8ee-289e685be324 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:02.427299 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab502d5-5c87-4417-b8ee-289e685be324 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:02.437869 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1405/2817] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:02.768319 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ab88a-f056-41e0-b7d6-26b8ed5e029e None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:02.769844 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3e7ab88a-f056-41e0-b7d6-26b8ed5e029e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:02.784049 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ab88a-f056-41e0-b7d6-26b8ed5e029e None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:02.788915 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0a8a2-b776-48d1-a037-ea929c898802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:02.789573 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0a8a2-b776-48d1-a037-ea929c898802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:02.789854 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0a8a2-b776-48d1-a037-ea929c898802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:02.790247 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0a8a2-b776-48d1-a037-ea929c898802 admin admin] PATH_INFO: `/v3/projects/1188b1283a7e4210885a9eb3b767b4d8` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:02.791443 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade0a8a2-b776-48d1-a037-ea929c898802 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1188b1283a7e4210885a9eb3b767b4d8)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:02.793409 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade0a8a2-b776-48d1-a037-ea929c898802 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:02.848507 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ade0a8a2-b776-48d1-a037-ea929c898802 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1188b1283a7e4210885a9eb3b767b4d8', 'request_id': 'req-ade0a8a2-b776-48d1-a037-ea929c898802'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:02.855932 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ade0a8a2-b776-48d1-a037-ea929c898802 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1188b1283a7e4210885a9eb3b767b4d8', 'request_id': 'req-ade0a8a2-b776-48d1-a037-ea929c898802'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:02.866983 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ade0a8a2-b776-48d1-a037-ea929c898802 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1188b1283a7e4210885a9eb3b767b4d8', 'request_id': 'req-ade0a8a2-b776-48d1-a037-ea929c898802'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:02.881424 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ade0a8a2-b776-48d1-a037-ea929c898802 admin admin] The token cache is being invalidate because project 1188b1283a7e4210885a9eb3b767b4d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:20:02.881816 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ade0a8a2-b776-48d1-a037-ea929c898802 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:02.882492 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ade0a8a2-b776-48d1-a037-ea929c898802 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:02.884441 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1413/2818] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:20:02 2022] DELETE /identity/v3/projects/1188b1283a7e4210885a9eb3b767b4d8 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:02.889043 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab502d5-5c87-4417-b8ee-289e685be324 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:02.899514 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab502d5-5c87-4417-b8ee-289e685be324 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:02.973485 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7790d30b-4b1b-485d-b1e7-d81c4c16253b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:02.974309 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790d30b-4b1b-485d-b1e7-d81c4c16253b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:02.974580 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790d30b-4b1b-485d-b1e7-d81c4c16253b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:02.974885 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790d30b-4b1b-485d-b1e7-d81c4c16253b admin admin] PATH_INFO: `/v3/users/221799880b9f472a879c371653f36af6` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:02.976236 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7790d30b-4b1b-485d-b1e7-d81c4c16253b admin admin] RBAC: Authorizing `identity:delete_user(user_id=221799880b9f472a879c371653f36af6)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:02.978363 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7790d30b-4b1b-485d-b1e7-d81c4c16253b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:03.039515 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-7790d30b-4b1b-485d-b1e7-d81c4c16253b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '221799880b9f472a879c371653f36af6'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:03.040127 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-7790d30b-4b1b-485d-b1e7-d81c4c16253b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '221799880b9f472a879c371653f36af6'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:03.052288 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-7790d30b-4b1b-485d-b1e7-d81c4c16253b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '221799880b9f472a879c371653f36af6'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:03.061493 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-7790d30b-4b1b-485d-b1e7-d81c4c16253b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '221799880b9f472a879c371653f36af6'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:03.074992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-7790d30b-4b1b-485d-b1e7-d81c4c16253b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '221799880b9f472a879c371653f36af6'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:03.081718 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1406/2819] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:20:02 2022] DELETE /identity/v3/users/221799880b9f472a879c371653f36af6 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:03.377738 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0a8a2-b776-48d1-a037-ea929c898802 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:03.379274 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-ade0a8a2-b776-48d1-a037-ea929c898802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:03.393590 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0a8a2-b776-48d1-a037-ea929c898802 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:03.439138 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:03.455330 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0836159-a203-4260-af79-1c820678c825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:03.455981 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0836159-a203-4260-af79-1c820678c825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:03.456263 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0836159-a203-4260-af79-1c820678c825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:03.456678 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0836159-a203-4260-af79-1c820678c825 admin admin] PATH_INFO: `/v3/projects/fef97eb490104876b65d08293255ddf5` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:03.457867 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0836159-a203-4260-af79-1c820678c825 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fef97eb490104876b65d08293255ddf5)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:03.459986 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0836159-a203-4260-af79-1c820678c825 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:03.512844 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e0836159-a203-4260-af79-1c820678c825 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fef97eb490104876b65d08293255ddf5', 'request_id': 'req-e0836159-a203-4260-af79-1c820678c825'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:03.518465 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e0836159-a203-4260-af79-1c820678c825 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fef97eb490104876b65d08293255ddf5', 'request_id': 'req-e0836159-a203-4260-af79-1c820678c825'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:03.530359 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e0836159-a203-4260-af79-1c820678c825 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fef97eb490104876b65d08293255ddf5', 'request_id': 'req-e0836159-a203-4260-af79-1c820678c825'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:03.547812 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e0836159-a203-4260-af79-1c820678c825 admin admin] The token cache is being invalidate because project fef97eb490104876b65d08293255ddf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:20:03.548087 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e0836159-a203-4260-af79-1c820678c825 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:03.548790 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-e0836159-a203-4260-af79-1c820678c825 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:03.550435 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1414/2820] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:20:03 2022] DELETE /identity/v3/projects/fef97eb490104876b65d08293255ddf5 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:03.555460 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f5d8e-d27b-410f-985c-7189ff1f3f6a None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:03.555980 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f5d8e-d27b-410f-985c-7189ff1f3f6a None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:03.556103 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f5d8e-d27b-410f-985c-7189ff1f3f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:03.576964 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-a49f5d8e-d27b-410f-985c-7189ff1f3f6a None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:03.578183 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-a49f5d8e-d27b-410f-985c-7189ff1f3f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:03.601346 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:03.650035 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1407/2821] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:20:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:03.658248 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941b1ba-6dee-41f7-84b2-3f068907ff4f None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:03.658525 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941b1ba-6dee-41f7-84b2-3f068907ff4f None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:03.658796 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941b1ba-6dee-41f7-84b2-3f068907ff4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:03.679928 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-0941b1ba-6dee-41f7-84b2-3f068907ff4f None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:03.681278 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-0941b1ba-6dee-41f7-84b2-3f068907ff4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:03.689902 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1415/2822] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:20:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:03.694647 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f5d8e-d27b-410f-985c-7189ff1f3f6a None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:03.710507 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f5d8e-d27b-410f-985c-7189ff1f3f6a None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:03.716950 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fcb3b-cb0b-47ec-bd92-1a41df581878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:03.717603 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fcb3b-cb0b-47ec-bd92-1a41df581878 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:03.717883 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fcb3b-cb0b-47ec-bd92-1a41df581878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:03.718152 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fcb3b-cb0b-47ec-bd92-1a41df581878 admin admin] PATH_INFO: `/v3/domains` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:03.718835 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73fcb3b-cb0b-47ec-bd92-1a41df581878 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:20:03.719137 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73fcb3b-cb0b-47ec-bd92-1a41df581878 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:03.720787 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73fcb3b-cb0b-47ec-bd92-1a41df581878 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:03.733345 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1408/2823] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:20:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:20:03.737951 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941b1ba-6dee-41f7-84b2-3f068907ff4f None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:03.747569 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941b1ba-6dee-41f7-84b2-3f068907ff4f None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:03.753132 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b00530-5097-43f3-9353-d5daef2f75dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:03.753747 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b00530-5097-43f3-9353-d5daef2f75dc admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:03.754023 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b00530-5097-43f3-9353-d5daef2f75dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:03.754309 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b00530-5097-43f3-9353-d5daef2f75dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:03.754998 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b00530-5097-43f3-9353-d5daef2f75dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:03.756952 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b00530-5097-43f3-9353-d5daef2f75dc admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:03.786851 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1416/2824] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:03 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:03.791409 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fcb3b-cb0b-47ec-bd92-1a41df581878 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:03.801970 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fcb3b-cb0b-47ec-bd92-1a41df581878 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:03.862693 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19f99c-1cc9-4245-9cb1-9729187b9cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:03.863325 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19f99c-1cc9-4245-9cb1-9729187b9cff admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:03.863597 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19f99c-1cc9-4245-9cb1-9729187b9cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:03.863903 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19f99c-1cc9-4245-9cb1-9729187b9cff admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:03.864607 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af19f99c-1cc9-4245-9cb1-9729187b9cff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:03.866549 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af19f99c-1cc9-4245-9cb1-9729187b9cff admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:03.891904 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1409/2825] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:03 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:03.896164 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b00530-5097-43f3-9353-d5daef2f75dc admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:03.907576 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b00530-5097-43f3-9353-d5daef2f75dc admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:03.913126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5544e6-49c2-4689-9c57-29e31b0d54b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:03.913784 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5544e6-49c2-4689-9c57-29e31b0d54b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:03.914050 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5544e6-49c2-4689-9c57-29e31b0d54b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:03.914340 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5544e6-49c2-4689-9c57-29e31b0d54b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:03.915164 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5544e6-49c2-4689-9c57-29e31b0d54b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:03.916845 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5544e6-49c2-4689-9c57-29e31b0d54b8 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:03.927799 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1417/2826] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:03.932937 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19f99c-1cc9-4245-9cb1-9729187b9cff admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:03.948649 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19f99c-1cc9-4245-9cb1-9729187b9cff admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:03.959373 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33060e-04e1-4992-8ee2-85bb0615aea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:03.960012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33060e-04e1-4992-8ee2-85bb0615aea2 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:03.960296 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33060e-04e1-4992-8ee2-85bb0615aea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:03.960585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33060e-04e1-4992-8ee2-85bb0615aea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:03.961306 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33060e-04e1-4992-8ee2-85bb0615aea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:03.962983 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33060e-04e1-4992-8ee2-85bb0615aea2 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:03.974621 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1410/2827] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:03.979723 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5544e6-49c2-4689-9c57-29e31b0d54b8 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:03.991180 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5544e6-49c2-4689-9c57-29e31b0d54b8 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:03.996118 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0da86e-6241-40e6-8f67-dc7aa06604c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:03.996855 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0da86e-6241-40e6-8f67-dc7aa06604c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:03.997127 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0da86e-6241-40e6-8f67-dc7aa06604c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:03.997431 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0da86e-6241-40e6-8f67-dc7aa06604c9 admin admin] PATH_INFO: `/v3/projects/a9cfbf38c749417f8f1488c59c23b1a0/users/2ea13d6d1ac1450b916246cff8c25f6d/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:04.014185 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0da86e-6241-40e6-8f67-dc7aa06604c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9cfbf38c749417f8f1488c59c23b1a0, user_id=2ea13d6d1ac1450b916246cff8c25f6d, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:04.016925 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0da86e-6241-40e6-8f67-dc7aa06604c9 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:04.033143 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1418/2828] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:03 2022] PUT /identity/v3/projects/a9cfbf38c749417f8f1488c59c23b1a0/users/2ea13d6d1ac1450b916246cff8c25f6d/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:04.045192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be447013-52d5-4792-b579-8387f43f1887 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:04.045476 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be447013-52d5-4792-b579-8387f43f1887 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:04.045747 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be447013-52d5-4792-b579-8387f43f1887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:04.076571 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-be447013-52d5-4792-b579-8387f43f1887 None None] MFA Rules not processed for user `2ea13d6d1ac1450b916246cff8c25f6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:04.077724 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-be447013-52d5-4792-b579-8387f43f1887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:04.161693 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1411/2829] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:20:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:04.166934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0da86e-6241-40e6-8f67-dc7aa06604c9 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:04.251146 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0da86e-6241-40e6-8f67-dc7aa06604c9 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:04.262057 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25b4dc-64ba-4510-bb65-11f9065f9ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:04.262984 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25b4dc-64ba-4510-bb65-11f9065f9ca1 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:04.263532 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25b4dc-64ba-4510-bb65-11f9065f9ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:04.264129 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25b4dc-64ba-4510-bb65-11f9065f9ca1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:04.265341 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a25b4dc-64ba-4510-bb65-11f9065f9ca1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:04.268777 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a25b4dc-64ba-4510-bb65-11f9065f9ca1 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:04.300104 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:04.311182 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1419/2830] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:04 2022] POST /identity/v3/projects => generated 381 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:04.315893 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be447013-52d5-4792-b579-8387f43f1887 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:04.331569 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be447013-52d5-4792-b579-8387f43f1887 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:04.352671 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:04.392949 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd367e2f-14dd-4e46-b6cd-3aeb5fe8104f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:04.393573 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd367e2f-14dd-4e46-b6cd-3aeb5fe8104f admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:04.393853 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd367e2f-14dd-4e46-b6cd-3aeb5fe8104f admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:04.394123 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd367e2f-14dd-4e46-b6cd-3aeb5fe8104f admin admin] PATH_INFO: `/v3/users` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:04.394788 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd367e2f-14dd-4e46-b6cd-3aeb5fe8104f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:04.396746 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd367e2f-14dd-4e46-b6cd-3aeb5fe8104f admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:04.423314 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1412/2831] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:04 2022] POST /identity/v3/users => generated 297 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:20:04.428789 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25b4dc-64ba-4510-bb65-11f9065f9ca1 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:04.441164 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25b4dc-64ba-4510-bb65-11f9065f9ca1 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:04.446318 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e78c0b-8367-48e1-9389-ba07e3c19196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:04.446986 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e78c0b-8367-48e1-9389-ba07e3c19196 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:04.447221 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e78c0b-8367-48e1-9389-ba07e3c19196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:04.447520 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e78c0b-8367-48e1-9389-ba07e3c19196 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:04.448219 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e78c0b-8367-48e1-9389-ba07e3c19196 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:04.449944 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e78c0b-8367-48e1-9389-ba07e3c19196 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:04.461529 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1420/2832] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:04.465900 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd367e2f-14dd-4e46-b6cd-3aeb5fe8104f admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:04.476189 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd367e2f-14dd-4e46-b6cd-3aeb5fe8104f admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:04.481392 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375e6e6-ae17-47e6-b804-fbe391b09abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:04.482072 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375e6e6-ae17-47e6-b804-fbe391b09abe admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:04.482354 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375e6e6-ae17-47e6-b804-fbe391b09abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:04.482636 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375e6e6-ae17-47e6-b804-fbe391b09abe admin admin] PATH_INFO: `/v3/projects/f18d706549c0458ea27ae26337a25ca0/users/c874418a60b34639af8c83cc32bde616/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:04.498636 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8375e6e6-ae17-47e6-b804-fbe391b09abe admin admin] RBAC: Authorizing `identity:create_grant(project_id=f18d706549c0458ea27ae26337a25ca0, user_id=c874418a60b34639af8c83cc32bde616, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:04.501361 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8375e6e6-ae17-47e6-b804-fbe391b09abe admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:04.516878 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1413/2833] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:04 2022] PUT /identity/v3/projects/f18d706549c0458ea27ae26337a25ca0/users/c874418a60b34639af8c83cc32bde616/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:04.521016 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e78c0b-8367-48e1-9389-ba07e3c19196 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:04.532033 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e78c0b-8367-48e1-9389-ba07e3c19196 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:04.593128 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8759cda-5e17-4765-910e-3f4aa986ad7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:04.593670 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8759cda-5e17-4765-910e-3f4aa986ad7d admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:04.593953 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8759cda-5e17-4765-910e-3f4aa986ad7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:04.594712 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8759cda-5e17-4765-910e-3f4aa986ad7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:04.594954 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8759cda-5e17-4765-910e-3f4aa986ad7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:04.596751 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8759cda-5e17-4765-910e-3f4aa986ad7d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:04.607684 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1421/2834] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:04 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:04.612735 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375e6e6-ae17-47e6-b804-fbe391b09abe admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:04.625009 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375e6e6-ae17-47e6-b804-fbe391b09abe admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:04.631352 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821b068-ce45-42e9-8470-616d490c1e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:04.632111 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821b068-ce45-42e9-8470-616d490c1e09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:04.632430 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821b068-ce45-42e9-8470-616d490c1e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:04.632745 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821b068-ce45-42e9-8470-616d490c1e09 admin admin] PATH_INFO: `/v3/projects/f18d706549c0458ea27ae26337a25ca0/users/c874418a60b34639af8c83cc32bde616/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:04.635238 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8821b068-ce45-42e9-8470-616d490c1e09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f18d706549c0458ea27ae26337a25ca0, user_id=c874418a60b34639af8c83cc32bde616, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:04.637995 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8821b068-ce45-42e9-8470-616d490c1e09 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:04.650304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1414/2835] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:04 2022] PUT /identity/v3/projects/f18d706549c0458ea27ae26337a25ca0/users/c874418a60b34639af8c83cc32bde616/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:04.661317 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a8102-374d-4c3b-9556-505518cba040 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:04.661811 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a8102-374d-4c3b-9556-505518cba040 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:04.662040 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a8102-374d-4c3b-9556-505518cba040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:04.691931 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-b27a8102-374d-4c3b-9556-505518cba040 None None] MFA Rules not processed for user `c874418a60b34639af8c83cc32bde616`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:04.693190 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-b27a8102-374d-4c3b-9556-505518cba040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:04.770148 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1422/2836] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:20:04 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:20:04.781986 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821b068-ce45-42e9-8470-616d490c1e09 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:04.784092 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-8821b068-ce45-42e9-8470-616d490c1e09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:04.849265 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821b068-ce45-42e9-8470-616d490c1e09 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:04.853777 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ec969-b4b4-4f1d-a603-28272b3d9c8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:04.854402 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ec969-b4b4-4f1d-a603-28272b3d9c8b service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:04.854682 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ec969-b4b4-4f1d-a603-28272b3d9c8b service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:04.854953 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ec969-b4b4-4f1d-a603-28272b3d9c8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:04.865407 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259ec969-b4b4-4f1d-a603-28272b3d9c8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:04.866853 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259ec969-b4b4-4f1d-a603-28272b3d9c8b service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:04.876992 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1415/2837] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:04 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:05.291524 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27a8102-374d-4c3b-9556-505518cba040 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:05.362624 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27a8102-374d-4c3b-9556-505518cba040 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:05.367770 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0ab4e-d28d-4e09-91da-b8c26aec01e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:05.368497 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0ab4e-d28d-4e09-91da-b8c26aec01e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:05.368779 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0ab4e-d28d-4e09-91da-b8c26aec01e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:05.369051 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0ab4e-d28d-4e09-91da-b8c26aec01e8 admin admin] PATH_INFO: `/v3/users/2ea13d6d1ac1450b916246cff8c25f6d` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:05.370257 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0ab4e-d28d-4e09-91da-b8c26aec01e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ea13d6d1ac1450b916246cff8c25f6d)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:05.372191 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0ab4e-d28d-4e09-91da-b8c26aec01e8 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:05.386535 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=100 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:05.429470 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4cb0ab4e-d28d-4e09-91da-b8c26aec01e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ea13d6d1ac1450b916246cff8c25f6d'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:05.430126 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4cb0ab4e-d28d-4e09-91da-b8c26aec01e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ea13d6d1ac1450b916246cff8c25f6d'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:05.451563 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4cb0ab4e-d28d-4e09-91da-b8c26aec01e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ea13d6d1ac1450b916246cff8c25f6d'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:05.464538 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4cb0ab4e-d28d-4e09-91da-b8c26aec01e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ea13d6d1ac1450b916246cff8c25f6d'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:05.465636 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-4cb0ab4e-d28d-4e09-91da-b8c26aec01e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ea13d6d1ac1450b916246cff8c25f6d'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:05.484325 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1423/2838] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:20:05 2022] DELETE /identity/v3/users/2ea13d6d1ac1450b916246cff8c25f6d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:05.490165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fccb3d-32ce-4b1a-b35c-91d6e83c9bb9 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:05.490458 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fccb3d-32ce-4b1a-b35c-91d6e83c9bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:05.490735 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fccb3d-32ce-4b1a-b35c-91d6e83c9bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:05.512716 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-51fccb3d-32ce-4b1a-b35c-91d6e83c9bb9 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:05.513966 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-51fccb3d-32ce-4b1a-b35c-91d6e83c9bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:05.584560 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1416/2839] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:20:05.595494 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0ab4e-d28d-4e09-91da-b8c26aec01e8 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:05.597043 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4cb0ab4e-d28d-4e09-91da-b8c26aec01e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:05.679791 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0ab4e-d28d-4e09-91da-b8c26aec01e8 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:05.683545 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bb8c2-a01d-43b9-b2a0-c763b4c97c18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:05.684175 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bb8c2-a01d-43b9-b2a0-c763b4c97c18 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:05.684488 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bb8c2-a01d-43b9-b2a0-c763b4c97c18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:05.684763 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bb8c2-a01d-43b9-b2a0-c763b4c97c18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:05.694396 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58bb8c2-a01d-43b9-b2a0-c763b4c97c18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:05.695821 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58bb8c2-a01d-43b9-b2a0-c763b4c97c18 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:05.706203 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1424/2840] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:06.015582 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fccb3d-32ce-4b1a-b35c-91d6e83c9bb9 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:06.017426 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-51fccb3d-32ce-4b1a-b35c-91d6e83c9bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:06.030883 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fccb3d-32ce-4b1a-b35c-91d6e83c9bb9 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:06.036310 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee8fc56-0b46-4ce3-b557-427c69d93054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:06.036990 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8fc56-0b46-4ce3-b557-427c69d93054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:06.037273 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8fc56-0b46-4ce3-b557-427c69d93054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:06.037575 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8fc56-0b46-4ce3-b557-427c69d93054 admin admin] PATH_INFO: `/v3/projects/a9cfbf38c749417f8f1488c59c23b1a0` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:06.038776 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee8fc56-0b46-4ce3-b557-427c69d93054 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9cfbf38c749417f8f1488c59c23b1a0)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:06.040822 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee8fc56-0b46-4ce3-b557-427c69d93054 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:06.054210 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:06.094822 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-0ee8fc56-0b46-4ce3-b557-427c69d93054 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9cfbf38c749417f8f1488c59c23b1a0', 'request_id': 'req-0ee8fc56-0b46-4ce3-b557-427c69d93054'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:06.102173 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-0ee8fc56-0b46-4ce3-b557-427c69d93054 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9cfbf38c749417f8f1488c59c23b1a0', 'request_id': 'req-0ee8fc56-0b46-4ce3-b557-427c69d93054'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:06.110969 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-0ee8fc56-0b46-4ce3-b557-427c69d93054 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9cfbf38c749417f8f1488c59c23b1a0', 'request_id': 'req-0ee8fc56-0b46-4ce3-b557-427c69d93054'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:06.123509 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-0ee8fc56-0b46-4ce3-b557-427c69d93054 admin admin] The token cache is being invalidate because project a9cfbf38c749417f8f1488c59c23b1a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:20:06.123795 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-0ee8fc56-0b46-4ce3-b557-427c69d93054 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:06.124581 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-0ee8fc56-0b46-4ce3-b557-427c69d93054 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:06.126259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1417/2841] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:20:06 2022] DELETE /identity/v3/projects/a9cfbf38c749417f8f1488c59c23b1a0 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:06.130561 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bb8c2-a01d-43b9-b2a0-c763b4c97c18 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:06.141851 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bb8c2-a01d-43b9-b2a0-c763b4c97c18 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:06.220893 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2f0ca0-7081-4743-8f7d-ef54a6852939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:06.221551 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f0ca0-7081-4743-8f7d-ef54a6852939 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:06.221882 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f0ca0-7081-4743-8f7d-ef54a6852939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:06.222106 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f0ca0-7081-4743-8f7d-ef54a6852939 admin admin] PATH_INFO: `/v3/users/c874418a60b34639af8c83cc32bde616` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:06.223355 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2f0ca0-7081-4743-8f7d-ef54a6852939 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c874418a60b34639af8c83cc32bde616)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:06.225351 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2f0ca0-7081-4743-8f7d-ef54a6852939 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:06.270591 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:06.285144 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ff2f0ca0-7081-4743-8f7d-ef54a6852939 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c874418a60b34639af8c83cc32bde616'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:06.285795 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ff2f0ca0-7081-4743-8f7d-ef54a6852939 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c874418a60b34639af8c83cc32bde616'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:06.300924 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ff2f0ca0-7081-4743-8f7d-ef54a6852939 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c874418a60b34639af8c83cc32bde616'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:06.309366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ff2f0ca0-7081-4743-8f7d-ef54a6852939 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c874418a60b34639af8c83cc32bde616'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:06.310198 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-ff2f0ca0-7081-4743-8f7d-ef54a6852939 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c874418a60b34639af8c83cc32bde616'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:06.325411 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1425/2842] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:20:06 2022] DELETE /identity/v3/users/c874418a60b34639af8c83cc32bde616 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:06.633708 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee8fc56-0b46-4ce3-b557-427c69d93054 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:06.635317 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-0ee8fc56-0b46-4ce3-b557-427c69d93054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:06.651312 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee8fc56-0b46-4ce3-b557-427c69d93054 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:06.728468 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49073ce-3f34-4fda-b25d-30473a8840ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:06.729577 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49073ce-3f34-4fda-b25d-30473a8840ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:06.730081 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49073ce-3f34-4fda-b25d-30473a8840ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:06.730562 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49073ce-3f34-4fda-b25d-30473a8840ff admin admin] PATH_INFO: `/v3/projects/f18d706549c0458ea27ae26337a25ca0` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:06.732674 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49073ce-3f34-4fda-b25d-30473a8840ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=f18d706549c0458ea27ae26337a25ca0)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:06.736198 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49073ce-3f34-4fda-b25d-30473a8840ff admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:06.803084 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a49073ce-3f34-4fda-b25d-30473a8840ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f18d706549c0458ea27ae26337a25ca0', 'request_id': 'req-a49073ce-3f34-4fda-b25d-30473a8840ff'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:06.809875 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a49073ce-3f34-4fda-b25d-30473a8840ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f18d706549c0458ea27ae26337a25ca0', 'request_id': 'req-a49073ce-3f34-4fda-b25d-30473a8840ff'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:06.819094 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a49073ce-3f34-4fda-b25d-30473a8840ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f18d706549c0458ea27ae26337a25ca0', 'request_id': 'req-a49073ce-3f34-4fda-b25d-30473a8840ff'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:06.831270 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a49073ce-3f34-4fda-b25d-30473a8840ff admin admin] The token cache is being invalidate because project f18d706549c0458ea27ae26337a25ca0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:20:06.831562 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a49073ce-3f34-4fda-b25d-30473a8840ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:06.832395 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-a49073ce-3f34-4fda-b25d-30473a8840ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:06.834155 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1418/2843] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:20:06 2022] DELETE /identity/v3/projects/f18d706549c0458ea27ae26337a25ca0 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:06.839839 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc17ff-18b4-4465-a1ae-2551866d6f92 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:06.840137 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc17ff-18b4-4465-a1ae-2551866d6f92 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:06.840466 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc17ff-18b4-4465-a1ae-2551866d6f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:06.862248 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c5dc17ff-18b4-4465-a1ae-2551866d6f92 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:06.863480 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c5dc17ff-18b4-4465-a1ae-2551866d6f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:06.936489 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1426/2844] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:20:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:20:06.945798 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ea797-8ee6-4366-8e9f-d84b033dca9d None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:06.946216 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ea797-8ee6-4366-8e9f-d84b033dca9d None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:06.946465 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ea797-8ee6-4366-8e9f-d84b033dca9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:06.971166 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-679ea797-8ee6-4366-8e9f-d84b033dca9d None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:06.972462 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-679ea797-8ee6-4366-8e9f-d84b033dca9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:06.982149 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1419/2845] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:20:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:06.987309 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc17ff-18b4-4465-a1ae-2551866d6f92 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:06.998195 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc17ff-18b4-4465-a1ae-2551866d6f92 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:07.003168 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e71420-da84-42d7-8b24-d0ec9e6b1d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:07.003795 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e71420-da84-42d7-8b24-d0ec9e6b1d24 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.004157 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e71420-da84-42d7-8b24-d0ec9e6b1d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.004469 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e71420-da84-42d7-8b24-d0ec9e6b1d24 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.005075 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e71420-da84-42d7-8b24-d0ec9e6b1d24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:20:07.005381 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e71420-da84-42d7-8b24-d0ec9e6b1d24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:07.006989 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e71420-da84-42d7-8b24-d0ec9e6b1d24 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:07.019947 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1427/2846] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:20:07.024710 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ea797-8ee6-4366-8e9f-d84b033dca9d None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:07.034767 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ea797-8ee6-4366-8e9f-d84b033dca9d None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:07.041154 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08724ba-0c5f-478c-a9e9-87a337ffb7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:07.042119 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08724ba-0c5f-478c-a9e9-87a337ffb7ab admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.042387 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08724ba-0c5f-478c-a9e9-87a337ffb7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.042662 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08724ba-0c5f-478c-a9e9-87a337ffb7ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.043748 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08724ba-0c5f-478c-a9e9-87a337ffb7ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:07.046953 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08724ba-0c5f-478c-a9e9-87a337ffb7ab admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:07.086897 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1420/2847] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:07 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:07.092896 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e71420-da84-42d7-8b24-d0ec9e6b1d24 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:07.107381 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e71420-da84-42d7-8b24-d0ec9e6b1d24 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:07.169493 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88784d87-afb6-42a2-a7ec-4dadc0996746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:07.170147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88784d87-afb6-42a2-a7ec-4dadc0996746 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.170417 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88784d87-afb6-42a2-a7ec-4dadc0996746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.170713 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88784d87-afb6-42a2-a7ec-4dadc0996746 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.171422 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88784d87-afb6-42a2-a7ec-4dadc0996746 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:07.173400 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88784d87-afb6-42a2-a7ec-4dadc0996746 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:07.199272 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1428/2848] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:07 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:07.202131 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08724ba-0c5f-478c-a9e9-87a337ffb7ab admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:07.207816 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:07.215398 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08724ba-0c5f-478c-a9e9-87a337ffb7ab admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:07.222437 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9227c-ab8d-4dfe-9d0b-ea99f97c1443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:07.223080 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9227c-ab8d-4dfe-9d0b-ea99f97c1443 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.223376 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9227c-ab8d-4dfe-9d0b-ea99f97c1443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.223651 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9227c-ab8d-4dfe-9d0b-ea99f97c1443 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.224407 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f9227c-ab8d-4dfe-9d0b-ea99f97c1443 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:07.226086 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f9227c-ab8d-4dfe-9d0b-ea99f97c1443 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:07.237807 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1421/2849] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:07.242008 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88784d87-afb6-42a2-a7ec-4dadc0996746 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:07.255253 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88784d87-afb6-42a2-a7ec-4dadc0996746 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:07.261649 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75611d-5fe5-4009-b34e-5bc3118f7db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:07.262352 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75611d-5fe5-4009-b34e-5bc3118f7db2 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.262655 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75611d-5fe5-4009-b34e-5bc3118f7db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.262998 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75611d-5fe5-4009-b34e-5bc3118f7db2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.263813 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f75611d-5fe5-4009-b34e-5bc3118f7db2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:07.265560 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f75611d-5fe5-4009-b34e-5bc3118f7db2 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:07.279427 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1429/2850] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:20:07.284791 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9227c-ab8d-4dfe-9d0b-ea99f97c1443 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:07.296815 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9227c-ab8d-4dfe-9d0b-ea99f97c1443 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:07.301972 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e1bd5f-b140-4621-9f4e-264a6755eb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:07.302657 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e1bd5f-b140-4621-9f4e-264a6755eb5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.302935 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e1bd5f-b140-4621-9f4e-264a6755eb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.303209 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e1bd5f-b140-4621-9f4e-264a6755eb5b admin admin] PATH_INFO: `/v3/projects/ad036d459c6449e18103177c2aaa1a17/users/d755ae29defd45c29e3c45aedba024ac/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.321821 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e1bd5f-b140-4621-9f4e-264a6755eb5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad036d459c6449e18103177c2aaa1a17, user_id=d755ae29defd45c29e3c45aedba024ac, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:07.324501 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e1bd5f-b140-4621-9f4e-264a6755eb5b admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:07.343249 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1422/2851] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:07 2022] PUT /identity/v3/projects/ad036d459c6449e18103177c2aaa1a17/users/d755ae29defd45c29e3c45aedba024ac/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:07.355951 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5d602-4e04-45a5-b69f-295be46e36e8 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.356534 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5d602-4e04-45a5-b69f-295be46e36e8 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.356828 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5d602-4e04-45a5-b69f-295be46e36e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.388961 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-c7c5d602-4e04-45a5-b69f-295be46e36e8 None None] MFA Rules not processed for user `d755ae29defd45c29e3c45aedba024ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:07.390228 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c7c5d602-4e04-45a5-b69f-295be46e36e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:07.394828 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:07.466651 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1430/2852] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:20:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:07.471376 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e1bd5f-b140-4621-9f4e-264a6755eb5b admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:07.539768 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e1bd5f-b140-4621-9f4e-264a6755eb5b admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:07.544886 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60a43b4-05e5-49f0-9c92-ebe3a5c0490e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:07.545626 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a43b4-05e5-49f0-9c92-ebe3a5c0490e admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.546589 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a43b4-05e5-49f0-9c92-ebe3a5c0490e admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.546589 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a43b4-05e5-49f0-9c92-ebe3a5c0490e admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.546924 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60a43b4-05e5-49f0-9c92-ebe3a5c0490e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:07.549028 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60a43b4-05e5-49f0-9c92-ebe3a5c0490e admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:07.579882 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1423/2853] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:07 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:07.584059 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5d602-4e04-45a5-b69f-295be46e36e8 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:07.595274 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5d602-4e04-45a5-b69f-295be46e36e8 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:07.664194 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c13e96-a809-4d93-8787-e3e7edc41fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:07.665230 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c13e96-a809-4d93-8787-e3e7edc41fa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.665541 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c13e96-a809-4d93-8787-e3e7edc41fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.665813 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c13e96-a809-4d93-8787-e3e7edc41fa5 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.666491 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c13e96-a809-4d93-8787-e3e7edc41fa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:07.668487 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c13e96-a809-4d93-8787-e3e7edc41fa5 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:07.693731 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1431/2854] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:07 2022] POST /identity/v3/users => generated 297 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:07.698007 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60a43b4-05e5-49f0-9c92-ebe3a5c0490e admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:07.709430 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60a43b4-05e5-49f0-9c92-ebe3a5c0490e admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:07.714924 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc73228-f387-46b8-9f0f-867036bac808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:07.715559 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc73228-f387-46b8-9f0f-867036bac808 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.715874 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc73228-f387-46b8-9f0f-867036bac808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.716145 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc73228-f387-46b8-9f0f-867036bac808 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.716883 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc73228-f387-46b8-9f0f-867036bac808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:07.718517 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc73228-f387-46b8-9f0f-867036bac808 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:07.730414 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1424/2855] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:07.735948 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c13e96-a809-4d93-8787-e3e7edc41fa5 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:07.749815 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c13e96-a809-4d93-8787-e3e7edc41fa5 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:07.755895 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f300a6-9512-482c-8323-2b6cb34470c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:07.756803 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f300a6-9512-482c-8323-2b6cb34470c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.757324 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f300a6-9512-482c-8323-2b6cb34470c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.757912 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f300a6-9512-482c-8323-2b6cb34470c7 admin admin] PATH_INFO: `/v3/projects/6b1d2ad650a44a99940a40614ddc118c/users/cdb9e2836f6448e69aaf1b9d6cb20795/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.775208 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f300a6-9512-482c-8323-2b6cb34470c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b1d2ad650a44a99940a40614ddc118c, user_id=cdb9e2836f6448e69aaf1b9d6cb20795, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:07.777980 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f300a6-9512-482c-8323-2b6cb34470c7 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:07.793897 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1432/2856] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:07 2022] PUT /identity/v3/projects/6b1d2ad650a44a99940a40614ddc118c/users/cdb9e2836f6448e69aaf1b9d6cb20795/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:07.798412 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc73228-f387-46b8-9f0f-867036bac808 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:07.809351 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc73228-f387-46b8-9f0f-867036bac808 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:07.875600 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ff27e-e580-48c4-b8a2-7660ff5f746e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:07.876226 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ff27e-e580-48c4-b8a2-7660ff5f746e admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.876678 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ff27e-e580-48c4-b8a2-7660ff5f746e admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.876831 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ff27e-e580-48c4-b8a2-7660ff5f746e admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.877502 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ff27e-e580-48c4-b8a2-7660ff5f746e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:07.879115 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ff27e-e580-48c4-b8a2-7660ff5f746e admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:07.890096 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1425/2857] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:07 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:07.894520 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f300a6-9512-482c-8323-2b6cb34470c7 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:07.905455 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f300a6-9512-482c-8323-2b6cb34470c7 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:07.912097 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b9ab3-3dd7-4ec9-acb7-50040b30ed69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:07.912800 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b9ab3-3dd7-4ec9-acb7-50040b30ed69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.913081 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b9ab3-3dd7-4ec9-acb7-50040b30ed69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.913348 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b9ab3-3dd7-4ec9-acb7-50040b30ed69 admin admin] PATH_INFO: `/v3/projects/6b1d2ad650a44a99940a40614ddc118c/users/cdb9e2836f6448e69aaf1b9d6cb20795/roles/3e1a9ff016c840c5b51ef7e3da9900ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.915647 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b9ab3-3dd7-4ec9-acb7-50040b30ed69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b1d2ad650a44a99940a40614ddc118c, user_id=cdb9e2836f6448e69aaf1b9d6cb20795, role_id=3e1a9ff016c840c5b51ef7e3da9900ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:07.918376 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b9ab3-3dd7-4ec9-acb7-50040b30ed69 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:07.931240 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1433/2858] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:07 2022] PUT /identity/v3/projects/6b1d2ad650a44a99940a40614ddc118c/users/cdb9e2836f6448e69aaf1b9d6cb20795/roles/3e1a9ff016c840c5b51ef7e3da9900ac => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:07.942193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f7e16-64b0-4a76-8027-8493ac6cbf52 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.942494 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f7e16-64b0-4a76-8027-8493ac6cbf52 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.942771 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f7e16-64b0-4a76-8027-8493ac6cbf52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.977166 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-cb8f7e16-64b0-4a76-8027-8493ac6cbf52 None None] MFA Rules not processed for user `cdb9e2836f6448e69aaf1b9d6cb20795`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:07.978499 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-cb8f7e16-64b0-4a76-8027-8493ac6cbf52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:08.011878 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:08.068028 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1426/2859] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:20:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:08.080454 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b9ab3-3dd7-4ec9-acb7-50040b30ed69 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:08.083420 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-1b9b9ab3-3dd7-4ec9-acb7-50040b30ed69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:08.151555 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b9ab3-3dd7-4ec9-acb7-50040b30ed69 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:08.155721 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e1bc11-d164-49ad-93ec-d4e91e276c7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:08.156724 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1bc11-d164-49ad-93ec-d4e91e276c7e service cinder] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:08.157007 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1bc11-d164-49ad-93ec-d4e91e276c7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:08.157280 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1bc11-d164-49ad-93ec-d4e91e276c7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:08.167702 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e1bc11-d164-49ad-93ec-d4e91e276c7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:08.169400 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e1bc11-d164-49ad-93ec-d4e91e276c7e service cinder] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:08.175233 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:08.181114 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1434/2860] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:08 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:08.262409 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8f7e16-64b0-4a76-8027-8493ac6cbf52 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:08.275269 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8f7e16-64b0-4a76-8027-8493ac6cbf52 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:08.280332 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f1b4d-02fd-40b4-9048-8b984cba133e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:08.280971 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f1b4d-02fd-40b4-9048-8b984cba133e service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:08.281251 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f1b4d-02fd-40b4-9048-8b984cba133e service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:08.281906 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f1b4d-02fd-40b4-9048-8b984cba133e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:08.362706 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f1b4d-02fd-40b4-9048-8b984cba133e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:08.364129 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f1b4d-02fd-40b4-9048-8b984cba133e service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:08.375016 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1427/2861] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:08.694360 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e1bc11-d164-49ad-93ec-d4e91e276c7e service cinder] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:08.769497 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e1bc11-d164-49ad-93ec-d4e91e276c7e service cinder] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:08.775977 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad601ef-810c-474e-a894-4e77aea69740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:08.776676 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad601ef-810c-474e-a894-4e77aea69740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:08.776955 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad601ef-810c-474e-a894-4e77aea69740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:08.777226 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad601ef-810c-474e-a894-4e77aea69740 admin admin] PATH_INFO: `/v3/users/d755ae29defd45c29e3c45aedba024ac` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:08.778512 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad601ef-810c-474e-a894-4e77aea69740 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d755ae29defd45c29e3c45aedba024ac)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:08.780497 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad601ef-810c-474e-a894-4e77aea69740 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:08.834449 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cad601ef-810c-474e-a894-4e77aea69740 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd755ae29defd45c29e3c45aedba024ac'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:08.835198 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cad601ef-810c-474e-a894-4e77aea69740 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd755ae29defd45c29e3c45aedba024ac'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:08.849973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cad601ef-810c-474e-a894-4e77aea69740 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd755ae29defd45c29e3c45aedba024ac'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:08.859158 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cad601ef-810c-474e-a894-4e77aea69740 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd755ae29defd45c29e3c45aedba024ac'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:08.859945 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-cad601ef-810c-474e-a894-4e77aea69740 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd755ae29defd45c29e3c45aedba024ac'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:08.874504 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1435/2862] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:20:08 2022] DELETE /identity/v3/users/d755ae29defd45c29e3c45aedba024ac => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:08.879225 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd16d1-b57b-4015-9df5-e733a6d699a3 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:08.879514 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd16d1-b57b-4015-9df5-e733a6d699a3 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:08.879788 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd16d1-b57b-4015-9df5-e733a6d699a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:08.899883 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-96fd16d1-b57b-4015-9df5-e733a6d699a3 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:08.901162 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-96fd16d1-b57b-4015-9df5-e733a6d699a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:08.965014 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1428/2863] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:20:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:08.977524 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad601ef-810c-474e-a894-4e77aea69740 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:08.978975 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-cad601ef-810c-474e-a894-4e77aea69740 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:09.040710 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad601ef-810c-474e-a894-4e77aea69740 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:09.044664 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79ca97-17a6-42a4-a12a-3cca589844f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:09.045279 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79ca97-17a6-42a4-a12a-3cca589844f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:09.045582 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79ca97-17a6-42a4-a12a-3cca589844f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:09.045837 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79ca97-17a6-42a4-a12a-3cca589844f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:09.056758 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef79ca97-17a6-42a4-a12a-3cca589844f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:09.058261 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef79ca97-17a6-42a4-a12a-3cca589844f0 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:09.069423 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1436/2864] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:09.381342 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd16d1-b57b-4015-9df5-e733a6d699a3 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:09.383328 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-96fd16d1-b57b-4015-9df5-e733a6d699a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:09.400070 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd16d1-b57b-4015-9df5-e733a6d699a3 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:09.407020 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c76e84-38f5-422b-a1b9-f633c4abde19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:09.408244 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c76e84-38f5-422b-a1b9-f633c4abde19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:09.408790 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c76e84-38f5-422b-a1b9-f633c4abde19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:09.409274 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c76e84-38f5-422b-a1b9-f633c4abde19 admin admin] PATH_INFO: `/v3/projects/ad036d459c6449e18103177c2aaa1a17` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:09.411611 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c76e84-38f5-422b-a1b9-f633c4abde19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad036d459c6449e18103177c2aaa1a17)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:09.415632 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c76e84-38f5-422b-a1b9-f633c4abde19 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:09.470819 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-71c76e84-38f5-422b-a1b9-f633c4abde19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad036d459c6449e18103177c2aaa1a17', 'request_id': 'req-71c76e84-38f5-422b-a1b9-f633c4abde19'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:09.477607 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-71c76e84-38f5-422b-a1b9-f633c4abde19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad036d459c6449e18103177c2aaa1a17', 'request_id': 'req-71c76e84-38f5-422b-a1b9-f633c4abde19'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:09.480534 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:09.486568 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-71c76e84-38f5-422b-a1b9-f633c4abde19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad036d459c6449e18103177c2aaa1a17', 'request_id': 'req-71c76e84-38f5-422b-a1b9-f633c4abde19'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:09.499438 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-71c76e84-38f5-422b-a1b9-f633c4abde19 admin admin] The token cache is being invalidate because project ad036d459c6449e18103177c2aaa1a17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:20:09.499763 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-71c76e84-38f5-422b-a1b9-f633c4abde19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:09.500470 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-71c76e84-38f5-422b-a1b9-f633c4abde19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:09.502036 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1429/2865] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:20:09 2022] DELETE /identity/v3/projects/ad036d459c6449e18103177c2aaa1a17 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:09.506296 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79ca97-17a6-42a4-a12a-3cca589844f0 service neutron] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:09.517381 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79ca97-17a6-42a4-a12a-3cca589844f0 service neutron] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:09.538715 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:09.587485 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3696a57-5c8a-4a7e-a928-e76051183b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:09.588157 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3696a57-5c8a-4a7e-a928-e76051183b20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:09.588495 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3696a57-5c8a-4a7e-a928-e76051183b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:09.588767 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3696a57-5c8a-4a7e-a928-e76051183b20 admin admin] PATH_INFO: `/v3/users/cdb9e2836f6448e69aaf1b9d6cb20795` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:09.589970 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3696a57-5c8a-4a7e-a928-e76051183b20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdb9e2836f6448e69aaf1b9d6cb20795)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:09.592011 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3696a57-5c8a-4a7e-a928-e76051183b20 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:09.641463 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a3696a57-5c8a-4a7e-a928-e76051183b20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdb9e2836f6448e69aaf1b9d6cb20795'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:09.642136 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a3696a57-5c8a-4a7e-a928-e76051183b20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdb9e2836f6448e69aaf1b9d6cb20795'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:09.657131 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a3696a57-5c8a-4a7e-a928-e76051183b20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdb9e2836f6448e69aaf1b9d6cb20795'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:09.664947 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a3696a57-5c8a-4a7e-a928-e76051183b20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdb9e2836f6448e69aaf1b9d6cb20795'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:09.665724 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-a3696a57-5c8a-4a7e-a928-e76051183b20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdb9e2836f6448e69aaf1b9d6cb20795'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:09.683840 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1437/2866] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:20:09 2022] DELETE /identity/v3/users/cdb9e2836f6448e69aaf1b9d6cb20795 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:10.041544 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c76e84-38f5-422b-a1b9-f633c4abde19 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:10.043275 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-71c76e84-38f5-422b-a1b9-f633c4abde19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:10.061174 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c76e84-38f5-422b-a1b9-f633c4abde19 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:10.130767 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f6435c-e135-42a3-8cb7-7c760c6648de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:10.131410 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6435c-e135-42a3-8cb7-7c760c6648de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:10.131692 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6435c-e135-42a3-8cb7-7c760c6648de admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:10.131965 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6435c-e135-42a3-8cb7-7c760c6648de admin admin] PATH_INFO: `/v3/projects/6b1d2ad650a44a99940a40614ddc118c` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:10.133255 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f6435c-e135-42a3-8cb7-7c760c6648de admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b1d2ad650a44a99940a40614ddc118c)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:10.135415 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f6435c-e135-42a3-8cb7-7c760c6648de admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:10.190107 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d5f6435c-e135-42a3-8cb7-7c760c6648de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b1d2ad650a44a99940a40614ddc118c', 'request_id': 'req-d5f6435c-e135-42a3-8cb7-7c760c6648de'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:10.197547 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d5f6435c-e135-42a3-8cb7-7c760c6648de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b1d2ad650a44a99940a40614ddc118c', 'request_id': 'req-d5f6435c-e135-42a3-8cb7-7c760c6648de'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:10.207217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d5f6435c-e135-42a3-8cb7-7c760c6648de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b1d2ad650a44a99940a40614ddc118c', 'request_id': 'req-d5f6435c-e135-42a3-8cb7-7c760c6648de'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:10.218122 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d5f6435c-e135-42a3-8cb7-7c760c6648de admin admin] The token cache is being invalidate because project 6b1d2ad650a44a99940a40614ddc118c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:20:10.218404 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d5f6435c-e135-42a3-8cb7-7c760c6648de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:10.219097 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-d5f6435c-e135-42a3-8cb7-7c760c6648de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:10.220682 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1430/2867] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:20:10 2022] DELETE /identity/v3/projects/6b1d2ad650a44a99940a40614ddc118c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:10.226392 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9dd26-1201-427f-94e7-1e23b2b38faa None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:10.226918 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9dd26-1201-427f-94e7-1e23b2b38faa None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:10.227015 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9dd26-1201-427f-94e7-1e23b2b38faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:10.251193 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-d5b9dd26-1201-427f-94e7-1e23b2b38faa None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:10.252616 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-d5b9dd26-1201-427f-94e7-1e23b2b38faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:10.328130 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1438/2868] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:20:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:10.337357 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2adc41-df7b-462d-afe3-02371e68ef97 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:10.337725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2adc41-df7b-462d-afe3-02371e68ef97 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:10.338062 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2adc41-df7b-462d-afe3-02371e68ef97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:10.360212 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-5b2adc41-df7b-462d-afe3-02371e68ef97 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:10.361462 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-5b2adc41-df7b-462d-afe3-02371e68ef97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:10.369724 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1431/2869] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:20:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:10.374428 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9dd26-1201-427f-94e7-1e23b2b38faa None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:10.385815 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9dd26-1201-427f-94e7-1e23b2b38faa None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:10.392265 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c441a41-75f4-414b-97ad-cb4ed8d8a389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:10.392923 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c441a41-75f4-414b-97ad-cb4ed8d8a389 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:10.393197 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c441a41-75f4-414b-97ad-cb4ed8d8a389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:10.393463 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c441a41-75f4-414b-97ad-cb4ed8d8a389 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:10.394159 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c441a41-75f4-414b-97ad-cb4ed8d8a389 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:20:10.394457 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c441a41-75f4-414b-97ad-cb4ed8d8a389 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:10.396061 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c441a41-75f4-414b-97ad-cb4ed8d8a389 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:10.409382 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1439/2870] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:20:10.414144 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2adc41-df7b-462d-afe3-02371e68ef97 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:10.424151 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2adc41-df7b-462d-afe3-02371e68ef97 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:10.429975 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d481071-a690-4f1d-80d9-040928fd394e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:10.430810 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d481071-a690-4f1d-80d9-040928fd394e admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:10.431087 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d481071-a690-4f1d-80d9-040928fd394e admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:10.431358 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d481071-a690-4f1d-80d9-040928fd394e admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:10.432040 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d481071-a690-4f1d-80d9-040928fd394e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:10.434163 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d481071-a690-4f1d-80d9-040928fd394e admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:10.464662 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1432/2871] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:10 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:20:10.469191 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c441a41-75f4-414b-97ad-cb4ed8d8a389 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:10.481020 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c441a41-75f4-414b-97ad-cb4ed8d8a389 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:10.511555 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:10.556031 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1684b-043d-4121-91ef-bcb3e4897b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:10.556729 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1684b-043d-4121-91ef-bcb3e4897b4d admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:10.557014 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1684b-043d-4121-91ef-bcb3e4897b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:10.557289 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1684b-043d-4121-91ef-bcb3e4897b4d admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:10.557973 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d1684b-043d-4121-91ef-bcb3e4897b4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:10.559937 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d1684b-043d-4121-91ef-bcb3e4897b4d admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:10.585790 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1440/2872] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:10 2022] POST /identity/v3/users => generated 288 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:10.590259 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d481071-a690-4f1d-80d9-040928fd394e admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:10.602070 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d481071-a690-4f1d-80d9-040928fd394e admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:10.607364 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e76d6-e51f-43a2-a2fd-6fe3e6e87273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:10.607999 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e76d6-e51f-43a2-a2fd-6fe3e6e87273 admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:10.608290 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e76d6-e51f-43a2-a2fd-6fe3e6e87273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:10.608584 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e76d6-e51f-43a2-a2fd-6fe3e6e87273 admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:10.609335 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e76d6-e51f-43a2-a2fd-6fe3e6e87273 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:10.611246 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e76d6-e51f-43a2-a2fd-6fe3e6e87273 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:10.623667 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1433/2873] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:10.627916 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1684b-043d-4121-91ef-bcb3e4897b4d admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:10.640184 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1684b-043d-4121-91ef-bcb3e4897b4d admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:10.646007 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5feaf4-022f-4a39-8bec-c3fb3cdbac0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:10.646652 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5feaf4-022f-4a39-8bec-c3fb3cdbac0a admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:10.646934 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5feaf4-022f-4a39-8bec-c3fb3cdbac0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:10.647212 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5feaf4-022f-4a39-8bec-c3fb3cdbac0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:10.647924 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5feaf4-022f-4a39-8bec-c3fb3cdbac0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:10.649624 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5feaf4-022f-4a39-8bec-c3fb3cdbac0a admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:10.662422 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1441/2874] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:10.667354 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e76d6-e51f-43a2-a2fd-6fe3e6e87273 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:10.678808 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e76d6-e51f-43a2-a2fd-6fe3e6e87273 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:10.683988 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9da886-2245-40f9-9b0e-0cae17bde403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:10.684725 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9da886-2245-40f9-9b0e-0cae17bde403 admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:10.685004 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9da886-2245-40f9-9b0e-0cae17bde403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:10.685271 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9da886-2245-40f9-9b0e-0cae17bde403 admin admin] PATH_INFO: `/v3/projects/09351810455346128fed3696def84e76/users/0d3d8017c05b4bd099bbcbf0cd120bc0/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:10.704718 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9da886-2245-40f9-9b0e-0cae17bde403 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09351810455346128fed3696def84e76, user_id=0d3d8017c05b4bd099bbcbf0cd120bc0, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:10.710217 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9da886-2245-40f9-9b0e-0cae17bde403 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:10.728347 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1434/2875] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:10 2022] PUT /identity/v3/projects/09351810455346128fed3696def84e76/users/0d3d8017c05b4bd099bbcbf0cd120bc0/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:20:10.741149 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94155bc-7caf-4373-b6c2-686bf4ba3419 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:10.741413 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94155bc-7caf-4373-b6c2-686bf4ba3419 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:10.741701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94155bc-7caf-4373-b6c2-686bf4ba3419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:10.772578 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-e94155bc-7caf-4373-b6c2-686bf4ba3419 None None] MFA Rules not processed for user `0d3d8017c05b4bd099bbcbf0cd120bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:10.773827 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-e94155bc-7caf-4373-b6c2-686bf4ba3419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:10.844807 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1442/2876] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:20:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:10.856532 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9da886-2245-40f9-9b0e-0cae17bde403 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:10.858614 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-3b9da886-2245-40f9-9b0e-0cae17bde403 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:10.874381 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:10.924494 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9da886-2245-40f9-9b0e-0cae17bde403 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:10.929062 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb77613-d33c-43d2-870e-86030d17c245 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:10.929691 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb77613-d33c-43d2-870e-86030d17c245 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:10.929965 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb77613-d33c-43d2-870e-86030d17c245 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:10.930239 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb77613-d33c-43d2-870e-86030d17c245 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:10.939192 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb77613-d33c-43d2-870e-86030d17c245 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:10.940710 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb77613-d33c-43d2-870e-86030d17c245 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:10.952543 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1435/2877] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:14.112612 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94155bc-7caf-4373-b6c2-686bf4ba3419 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:14.189006 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94155bc-7caf-4373-b6c2-686bf4ba3419 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:14.195038 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3087355c-2689-4a09-8cf4-fbd83f45a200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:14.195709 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087355c-2689-4a09-8cf4-fbd83f45a200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:14.195988 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087355c-2689-4a09-8cf4-fbd83f45a200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:14.196261 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087355c-2689-4a09-8cf4-fbd83f45a200 admin admin] PATH_INFO: `/v3/users/0d3d8017c05b4bd099bbcbf0cd120bc0` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:14.197630 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3087355c-2689-4a09-8cf4-fbd83f45a200 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d3d8017c05b4bd099bbcbf0cd120bc0)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:14.199632 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3087355c-2689-4a09-8cf4-fbd83f45a200 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:14.247909 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:14.254905 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3087355c-2689-4a09-8cf4-fbd83f45a200 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d3d8017c05b4bd099bbcbf0cd120bc0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:14.256648 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3087355c-2689-4a09-8cf4-fbd83f45a200 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d3d8017c05b4bd099bbcbf0cd120bc0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:14.272005 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3087355c-2689-4a09-8cf4-fbd83f45a200 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d3d8017c05b4bd099bbcbf0cd120bc0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:14.279293 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3087355c-2689-4a09-8cf4-fbd83f45a200 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d3d8017c05b4bd099bbcbf0cd120bc0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:14.280500 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-3087355c-2689-4a09-8cf4-fbd83f45a200 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d3d8017c05b4bd099bbcbf0cd120bc0'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:14.296382 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1443/2878] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:20:14 2022] DELETE /identity/v3/users/0d3d8017c05b4bd099bbcbf0cd120bc0 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:14.301258 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec719d3e-1709-409a-9faf-f72e57f417b3 None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:14.301871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec719d3e-1709-409a-9faf-f72e57f417b3 None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:14.301871 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec719d3e-1709-409a-9faf-f72e57f417b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:14.323003 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-ec719d3e-1709-409a-9faf-f72e57f417b3 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:14.324254 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ec719d3e-1709-409a-9faf-f72e57f417b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:14.391172 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1436/2879] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:20:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:14.401537 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3087355c-2689-4a09-8cf4-fbd83f45a200 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:14.403101 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3087355c-2689-4a09-8cf4-fbd83f45a200 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:14.472163 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3087355c-2689-4a09-8cf4-fbd83f45a200 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:14.478549 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1390919f-4166-411d-8eba-fa6878c1dc25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:14.479479 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390919f-4166-411d-8eba-fa6878c1dc25 service neutron] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:14.479913 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390919f-4166-411d-8eba-fa6878c1dc25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:14.480368 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390919f-4166-411d-8eba-fa6878c1dc25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:14.499465 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1390919f-4166-411d-8eba-fa6878c1dc25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:14.501486 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1390919f-4166-411d-8eba-fa6878c1dc25 service neutron] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:14.515813 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1444/2880] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:14.822852 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec719d3e-1709-409a-9faf-f72e57f417b3 None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:14.824741 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-ec719d3e-1709-409a-9faf-f72e57f417b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:14.837680 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec719d3e-1709-409a-9faf-f72e57f417b3 None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:14.843376 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c7b3f-00cc-4644-90ab-231b8419f825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:14.844140 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c7b3f-00cc-4644-90ab-231b8419f825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:14.844597 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c7b3f-00cc-4644-90ab-231b8419f825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:14.844930 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c7b3f-00cc-4644-90ab-231b8419f825 admin admin] PATH_INFO: `/v3/projects/09351810455346128fed3696def84e76` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:14.846249 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9c7b3f-00cc-4644-90ab-231b8419f825 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09351810455346128fed3696def84e76)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:14.848193 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9c7b3f-00cc-4644-90ab-231b8419f825 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:14.898356 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-0a9c7b3f-00cc-4644-90ab-231b8419f825 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09351810455346128fed3696def84e76', 'request_id': 'req-0a9c7b3f-00cc-4644-90ab-231b8419f825'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:14.905372 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-0a9c7b3f-00cc-4644-90ab-231b8419f825 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09351810455346128fed3696def84e76', 'request_id': 'req-0a9c7b3f-00cc-4644-90ab-231b8419f825'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:14.914693 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-0a9c7b3f-00cc-4644-90ab-231b8419f825 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09351810455346128fed3696def84e76', 'request_id': 'req-0a9c7b3f-00cc-4644-90ab-231b8419f825'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:14.925846 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-0a9c7b3f-00cc-4644-90ab-231b8419f825 admin admin] The token cache is being invalidate because project 09351810455346128fed3696def84e76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:20:14.926127 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-0a9c7b3f-00cc-4644-90ab-231b8419f825 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:14.926777 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-0a9c7b3f-00cc-4644-90ab-231b8419f825 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:14.928322 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1437/2881] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:20:14 2022] DELETE /identity/v3/projects/09351810455346128fed3696def84e76 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:14.934250 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44c729-56a0-4634-b4c0-40563af3ab9d None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:14.934605 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44c729-56a0-4634-b4c0-40563af3ab9d None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:14.934953 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44c729-56a0-4634-b4c0-40563af3ab9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:14.963455 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-3b44c729-56a0-4634-b4c0-40563af3ab9d None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:14.964701 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-3b44c729-56a0-4634-b4c0-40563af3ab9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:15.027331 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1445/2882] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:20:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:15.036850 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c762431-2ab1-4aed-af20-0b6ddb9e088d None None] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:15.037131 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c762431-2ab1-4aed-af20-0b6ddb9e088d None None] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:15.037410 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c762431-2ab1-4aed-af20-0b6ddb9e088d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:15.057727 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.auth.core [None req-7c762431-2ab1-4aed-af20-0b6ddb9e088d None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:15.058952 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-7c762431-2ab1-4aed-af20-0b6ddb9e088d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:15.073492 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1438/2883] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:20:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:15.079792 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44c729-56a0-4634-b4c0-40563af3ab9d None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:15.090515 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44c729-56a0-4634-b4c0-40563af3ab9d None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:15.096134 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0046a315-23fd-4c99-a3ab-f09b5491b610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:15.096800 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0046a315-23fd-4c99-a3ab-f09b5491b610 admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:15.097078 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0046a315-23fd-4c99-a3ab-f09b5491b610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:15.097352 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0046a315-23fd-4c99-a3ab-f09b5491b610 admin admin] PATH_INFO: `/v3/domains` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:15.098052 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0046a315-23fd-4c99-a3ab-f09b5491b610 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:20:15.098349 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0046a315-23fd-4c99-a3ab-f09b5491b610 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:15.100044 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0046a315-23fd-4c99-a3ab-f09b5491b610 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:15.112734 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1446/2884] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 14:20:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:20:15.117263 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c762431-2ab1-4aed-af20-0b6ddb9e088d None None] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:15.127081 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c762431-2ab1-4aed-af20-0b6ddb9e088d None None] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:15.133087 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080fb52-a638-4c5d-aa60-8d277ca5d067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:15.133777 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080fb52-a638-4c5d-aa60-8d277ca5d067 admin admin] REQUEST_METHOD: `POST` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:15.134058 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080fb52-a638-4c5d-aa60-8d277ca5d067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:15.134331 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080fb52-a638-4c5d-aa60-8d277ca5d067 admin admin] PATH_INFO: `/v3/projects` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:15.135009 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d080fb52-a638-4c5d-aa60-8d277ca5d067 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:15.137041 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d080fb52-a638-4c5d-aa60-8d277ca5d067 admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:15.170052 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1439/2885] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:15 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:15.174608 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0046a315-23fd-4c99-a3ab-f09b5491b610 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:15.185121 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0046a315-23fd-4c99-a3ab-f09b5491b610 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:15.230895 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:15.248239 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e535ef-7ce8-46ec-ae79-426f7369b5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:15.248894 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e535ef-7ce8-46ec-ae79-426f7369b5f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:15.249176 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e535ef-7ce8-46ec-ae79-426f7369b5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:15.249470 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e535ef-7ce8-46ec-ae79-426f7369b5f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:15.250142 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e535ef-7ce8-46ec-ae79-426f7369b5f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:15.252120 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e535ef-7ce8-46ec-ae79-426f7369b5f3 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:15.274842 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1447/2886] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:15 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:20:15.279323 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080fb52-a638-4c5d-aa60-8d277ca5d067 admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:15.290652 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080fb52-a638-4c5d-aa60-8d277ca5d067 admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:15.296979 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a853c8ca-a891-41bd-85eb-65666471ce7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:15.297585 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a853c8ca-a891-41bd-85eb-65666471ce7d admin admin] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:15.297868 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a853c8ca-a891-41bd-85eb-65666471ce7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:15.298134 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a853c8ca-a891-41bd-85eb-65666471ce7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:15.298824 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a853c8ca-a891-41bd-85eb-65666471ce7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:15.300470 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a853c8ca-a891-41bd-85eb-65666471ce7d admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:15.302958 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:15.313070 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1440/2887] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:15.317341 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e535ef-7ce8-46ec-ae79-426f7369b5f3 admin admin] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:15.328764 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e535ef-7ce8-46ec-ae79-426f7369b5f3 admin admin] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:15.333800 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7864c75-9db4-442b-942c-078c63d150ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:15.334430 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7864c75-9db4-442b-942c-078c63d150ce admin admin] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:15.334703 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7864c75-9db4-442b-942c-078c63d150ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:15.334977 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7864c75-9db4-442b-942c-078c63d150ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:15.335676 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7864c75-9db4-442b-942c-078c63d150ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:15.337343 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7864c75-9db4-442b-942c-078c63d150ce admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:15.352208 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1448/2888] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:15.357329 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a853c8ca-a891-41bd-85eb-65666471ce7d admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:15.368427 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a853c8ca-a891-41bd-85eb-65666471ce7d admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:15.375683 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcac843-ba64-4bac-9554-2bb9f0772c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:15.376917 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcac843-ba64-4bac-9554-2bb9f0772c2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:15.377415 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcac843-ba64-4bac-9554-2bb9f0772c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:15.377890 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcac843-ba64-4bac-9554-2bb9f0772c2e admin admin] PATH_INFO: `/v3/projects/192684124f13419fa3f9aed76ac3e400/users/3ad71c665f4f4aebb823e22b4e60a3a2/roles/75aff8cb4cf748a2adbd6c3f141fe8ac` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:15.399829 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcac843-ba64-4bac-9554-2bb9f0772c2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=192684124f13419fa3f9aed76ac3e400, user_id=3ad71c665f4f4aebb823e22b4e60a3a2, role_id=75aff8cb4cf748a2adbd6c3f141fe8ac)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:15.402553 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcac843-ba64-4bac-9554-2bb9f0772c2e admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:15.417398 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1441/2889] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:15 2022] PUT /identity/v3/projects/192684124f13419fa3f9aed76ac3e400/users/3ad71c665f4f4aebb823e22b4e60a3a2/roles/75aff8cb4cf748a2adbd6c3f141fe8ac => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:15.423409 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d188a8-9117-43e4-8521-c52fa0b85f16 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:15.423708 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d188a8-9117-43e4-8521-c52fa0b85f16 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:15.424021 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d188a8-9117-43e4-8521-c52fa0b85f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:15.445137 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-a9d188a8-9117-43e4-8521-c52fa0b85f16 None None] MFA Rules not processed for user `bf301c0e8be34b7fa129e68f7514a379`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:15.446589 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-a9d188a8-9117-43e4-8521-c52fa0b85f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:15.520133 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1449/2890] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:20:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:15.533034 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcac843-ba64-4bac-9554-2bb9f0772c2e admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:15.535319 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-2dcac843-ba64-4bac-9554-2bb9f0772c2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:15.618207 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcac843-ba64-4bac-9554-2bb9f0772c2e admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:15.622012 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b05501-c118-4047-8af0-12897739f012 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:15.622639 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b05501-c118-4047-8af0-12897739f012 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:15.622922 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b05501-c118-4047-8af0-12897739f012 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:15.623197 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b05501-c118-4047-8af0-12897739f012 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:15.633451 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b05501-c118-4047-8af0-12897739f012 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:15.634894 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b05501-c118-4047-8af0-12897739f012 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:15.644911 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1442/2891] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:16.614344 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08c98c-159b-448f-82c4-0cde273fa1cb None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:16.615220 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08c98c-159b-448f-82c4-0cde273fa1cb None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:16.615220 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08c98c-159b-448f-82c4-0cde273fa1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:16.651429 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-4c08c98c-159b-448f-82c4-0cde273fa1cb None None] MFA Rules not processed for user `3ad71c665f4f4aebb823e22b4e60a3a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:16.652725 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4c08c98c-159b-448f-82c4-0cde273fa1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:16.741098 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1450/2892] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:20:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:16.748925 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b05501-c118-4047-8af0-12897739f012 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:16.750467 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-62b05501-c118-4047-8af0-12897739f012 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:16.816243 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b05501-c118-4047-8af0-12897739f012 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:16.820168 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4064f-7308-4704-a4fc-675218e7fba7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4c3d46ac7243aeb722a4410f40d846', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:16.820929 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4064f-7308-4704-a4fc-675218e7fba7 service cinder] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:16.821210 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4064f-7308-4704-a4fc-675218e7fba7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:16.821482 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4064f-7308-4704-a4fc-675218e7fba7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:16.831310 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff4064f-7308-4704-a4fc-675218e7fba7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:16.832783 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff4064f-7308-4704-a4fc-675218e7fba7 service cinder] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:16.843383 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1443/2893] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:18.244621 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc5a8f-f120-4b82-9ca1-f875a6035fbd None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:18.245255 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc5a8f-f120-4b82-9ca1-f875a6035fbd None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:18.245255 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc5a8f-f120-4b82-9ca1-f875a6035fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:18.267064 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-58dc5a8f-f120-4b82-9ca1-f875a6035fbd None None] MFA Rules not processed for user `3ad71c665f4f4aebb823e22b4e60a3a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:18.268246 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-58dc5a8f-f120-4b82-9ca1-f875a6035fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:18.276133 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1451/2894] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:20:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:20:18.554946 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4064f-7308-4704-a4fc-675218e7fba7 service cinder] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:18.556711 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-dff4064f-7308-4704-a4fc-675218e7fba7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:18.570625 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4064f-7308-4704-a4fc-675218e7fba7 service cinder] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:18.574918 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34cf954-db88-4c68-8a5f-bcde001fe487 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dd44b7a78594288b6aabb393fd6324a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:18.575568 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34cf954-db88-4c68-8a5f-bcde001fe487 service neutron] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:18.575849 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34cf954-db88-4c68-8a5f-bcde001fe487 service neutron] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:18.576126 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34cf954-db88-4c68-8a5f-bcde001fe487 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:18.578157 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c34cf954-db88-4c68-8a5f-bcde001fe487 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:18.589496 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34cf954-db88-4c68-8a5f-bcde001fe487 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:18.590964 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34cf954-db88-4c68-8a5f-bcde001fe487 service neutron] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:18.600237 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1444/2895] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:19.955531 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc5a8f-f120-4b82-9ca1-f875a6035fbd None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:19.957029 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-58dc5a8f-f120-4b82-9ca1-f875a6035fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:20.014178 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:20.047231 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc5a8f-f120-4b82-9ca1-f875a6035fbd None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:20.055346 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e26263-3102-4891-8936-33b92005baa4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58380e30789041e4ad9e8652bc628658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:20.056343 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e26263-3102-4891-8936-33b92005baa4 service placement] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:20.056786 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e26263-3102-4891-8936-33b92005baa4 service placement] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:20.057213 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e26263-3102-4891-8936-33b92005baa4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:20.059870 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-f7e26263-3102-4891-8936-33b92005baa4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:20.073918 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e26263-3102-4891-8936-33b92005baa4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:20.075523 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e26263-3102-4891-8936-33b92005baa4 service placement] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:20.087596 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1452/2896] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:28.596293 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=69,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:29.938917 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34cf954-db88-4c68-8a5f-bcde001fe487 service neutron] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:29.942068 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-c34cf954-db88-4c68-8a5f-bcde001fe487 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:30.083392 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=10 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:30.096923 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34cf954-db88-4c68-8a5f-bcde001fe487 service neutron] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:30.103320 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732872b-1237-46a5-8cd3-ae5f5de5d66d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3adc14dcaf4ac79b43bb97fa1ba1d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:30.104062 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732872b-1237-46a5-8cd3-ae5f5de5d66d service nova] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:30.104425 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732872b-1237-46a5-8cd3-ae5f5de5d66d service nova] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:30.104801 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732872b-1237-46a5-8cd3-ae5f5de5d66d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:30.106969 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-4732872b-1237-46a5-8cd3-ae5f5de5d66d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:30.122912 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4732872b-1237-46a5-8cd3-ae5f5de5d66d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:30.126119 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4732872b-1237-46a5-8cd3-ae5f5de5d66d service nova] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:30.141359 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1445/2897] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:40.135837 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=34 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:22.470366 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210b34a-d21f-4569-adcb-be589f31f704 None None] REQUEST_METHOD: `POST` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:22.471174 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210b34a-d21f-4569-adcb-be589f31f704 None None] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:22.471174 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210b34a-d21f-4569-adcb-be589f31f704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:22.494012 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.auth.core [None req-4210b34a-d21f-4569-adcb-be589f31f704 None None] MFA Rules not processed for user `3ad71c665f4f4aebb823e22b4e60a3a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:22.495275 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-4210b34a-d21f-4569-adcb-be589f31f704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:22.503147 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1453/2898] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 14:21:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:32.487861 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=3 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:54.470840 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732872b-1237-46a5-8cd3-ae5f5de5d66d service nova] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:54.472304 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-4732872b-1237-46a5-8cd3-ae5f5de5d66d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:54.486475 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732872b-1237-46a5-8cd3-ae5f5de5d66d service nova] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:54.491979 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a00ae-7736-46f6-8611-9feb160be6fe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58380e30789041e4ad9e8652bc628658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:54.492629 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a00ae-7736-46f6-8611-9feb160be6fe service placement] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:54.493032 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a00ae-7736-46f6-8611-9feb160be6fe service placement] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:54.493169 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a00ae-7736-46f6-8611-9feb160be6fe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:54.495009 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-044a00ae-7736-46f6-8611-9feb160be6fe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:54.507582 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a00ae-7736-46f6-8611-9feb160be6fe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:54.509214 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a00ae-7736-46f6-8611-9feb160be6fe service placement] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:54.519988 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1446/2899] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:22:04.515165 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=6 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:17.689805 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210b34a-d21f-4569-adcb-be589f31f704 None None] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:17.703371 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210b34a-d21f-4569-adcb-be589f31f704 None None] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:17.709460 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c770958a-ac6d-47d3-987d-767bcc55f25a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58380e30789041e4ad9e8652bc628658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:17.710127 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770958a-ac6d-47d3-987d-767bcc55f25a service placement] REQUEST_METHOD: `GET` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:17.710383 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770958a-ac6d-47d3-987d-767bcc55f25a service placement] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:17.710647 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770958a-ac6d-47d3-987d-767bcc55f25a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:17.712538 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.fernet_utils [None req-c770958a-ac6d-47d3-987d-767bcc55f25a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:17.726707 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c770958a-ac6d-47d3-987d-767bcc55f25a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:17.728558 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c770958a-ac6d-47d3-987d-767bcc55f25a service placement] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:17.741197 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1454/2900] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:22:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:22:17.756787 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a00ae-7736-46f6-8611-9feb160be6fe service placement] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:17.776065 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a00ae-7736-46f6-8611-9feb160be6fe service placement] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:17.783188 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aeb0e1-72b4-4f50-a28e-7640e30dc34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:17.783811 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aeb0e1-72b4-4f50-a28e-7640e30dc34c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:17.784102 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aeb0e1-72b4-4f50-a28e-7640e30dc34c admin admin] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:17.784442 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aeb0e1-72b4-4f50-a28e-7640e30dc34c admin admin] PATH_INFO: `/v3/users/3ad71c665f4f4aebb823e22b4e60a3a2` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:17.786086 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aeb0e1-72b4-4f50-a28e-7640e30dc34c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ad71c665f4f4aebb823e22b4e60a3a2)` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:17.788489 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aeb0e1-72b4-4f50-a28e-7640e30dc34c admin admin] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:17.847291 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-42aeb0e1-72b4-4f50-a28e-7640e30dc34c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ad71c665f4f4aebb823e22b4e60a3a2'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:17.847984 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-42aeb0e1-72b4-4f50-a28e-7640e30dc34c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ad71c665f4f4aebb823e22b4e60a3a2'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:17.856651 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-42aeb0e1-72b4-4f50-a28e-7640e30dc34c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ad71c665f4f4aebb823e22b4e60a3a2'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:17.863900 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-42aeb0e1-72b4-4f50-a28e-7640e30dc34c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ad71c665f4f4aebb823e22b4e60a3a2'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:17.878077 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.notifications [None req-42aeb0e1-72b4-4f50-a28e-7640e30dc34c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ad71c665f4f4aebb823e22b4e60a3a2'} {{(pid=80310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:17.885211 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1447/2901] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 14:22:17 2022] DELETE /identity/v3/users/3ad71c665f4f4aebb823e22b4e60a3a2 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:22:18.077960 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c770958a-ac6d-47d3-987d-767bcc55f25a service placement] Authenticating user token {{(pid=80311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:18.089313 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c770958a-ac6d-47d3-987d-767bcc55f25a service placement] Validating token access rules against request {{(pid=80311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:18.149444 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dad38a-a105-47ee-a47c-bb7530462d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf301c0e8be34b7fa129e68f7514a379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a12613c4ac44ad788edb767368b347b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:18.150083 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dad38a-a105-47ee-a47c-bb7530462d19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:18.150357 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dad38a-a105-47ee-a47c-bb7530462d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:18.150628 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dad38a-a105-47ee-a47c-bb7530462d19 admin admin] PATH_INFO: `/v3/projects/192684124f13419fa3f9aed76ac3e400` {{(pid=80311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:18.151903 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dad38a-a105-47ee-a47c-bb7530462d19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=192684124f13419fa3f9aed76ac3e400)` {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:18.153909 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dad38a-a105-47ee-a47c-bb7530462d19 admin admin] RBAC: Authorization granted {{(pid=80311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:18.213229 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-54dad38a-a105-47ee-a47c-bb7530462d19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '192684124f13419fa3f9aed76ac3e400', 'request_id': 'req-54dad38a-a105-47ee-a47c-bb7530462d19'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:18.218551 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-54dad38a-a105-47ee-a47c-bb7530462d19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '192684124f13419fa3f9aed76ac3e400', 'request_id': 'req-54dad38a-a105-47ee-a47c-bb7530462d19'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:18.226848 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-54dad38a-a105-47ee-a47c-bb7530462d19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '192684124f13419fa3f9aed76ac3e400', 'request_id': 'req-54dad38a-a105-47ee-a47c-bb7530462d19'} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:18.238772 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-54dad38a-a105-47ee-a47c-bb7530462d19 admin admin] The token cache is being invalidate because project 192684124f13419fa3f9aed76ac3e400 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:22:18.239078 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-54dad38a-a105-47ee-a47c-bb7530462d19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:18.239735 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG keystone.notifications [None req-54dad38a-a105-47ee-a47c-bb7530462d19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:18.241316 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: [pid: 80311|app: 0|req: 1455/2902] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 14:22:18 2022] DELETE /identity/v3/projects/192684124f13419fa3f9aed76ac3e400 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:22:27.881167 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG dbcounter [-] [80310] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:28.235370 dv-cinder-845178-10-iscsi devstack@keystone.service[80311]: DEBUG dbcounter [-] [80311] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:43.429338 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aeb0e1-72b4-4f50-a28e-7640e30dc34c admin admin] Authenticating user token {{(pid=80310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:43.431350 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-42aeb0e1-72b4-4f50-a28e-7640e30dc34c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:43.597034 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aeb0e1-72b4-4f50-a28e-7640e30dc34c admin admin] Validating token access rules against request {{(pid=80310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:43.661652 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ff8c05-fffd-474e-bb7d-8aca6136e06d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58380e30789041e4ad9e8652bc628658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36f063d8f65d494a89dbbd72c53417c8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:43.662206 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff8c05-fffd-474e-bb7d-8aca6136e06d service placement] REQUEST_METHOD: `GET` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:43.662514 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff8c05-fffd-474e-bb7d-8aca6136e06d service placement] SCRIPT_NAME: `/identity` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:43.662762 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff8c05-fffd-474e-bb7d-8aca6136e06d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:43.664668 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.fernet_utils [None req-88ff8c05-fffd-474e-bb7d-8aca6136e06d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:43.676294 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ff8c05-fffd-474e-bb7d-8aca6136e06d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:43.677828 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ff8c05-fffd-474e-bb7d-8aca6136e06d service placement] RBAC: Authorization granted {{(pid=80310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:43.688046 dv-cinder-845178-10-iscsi devstack@keystone.service[80310]: [pid: 80310|app: 0|req: 1448/2903] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 14:27:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)